Bob Atkey's blog

More Data Types with Negation at Fun in the REPL

Thu, 02 Nov 2023 00:00:00 +0000

Yesterday I had a lovely day at Fun in the REPL in Bristol.

Alex Kavvos invited me to give a talk, so I decided to give an updated and extended version of the Data types with Negation talk I gave a few times last year.

I've got a bit further in working out how to analyse the semantics of a data type with negation, but seeing an analogy with the solutions of mixed-variance domain equations.

Here are the slides I used:

Compiling higher-order specifications to SMT solvers

Tue, 17 Jan 2023 00:00:00 +0000

I just gave a talk at Certified Programs and Proofs on our paper Compiling higher-order specifications to SMT solvers.

Slides:

It is a description of the type based analysis approach that we use in the Vehicle specification language to discover when the queries that a user writes are actually translatable to an SMT solver. Based on this type based analysis we either give the user a nice error message, or we use a Normalisation by Evaluation (NbE) procedure to normalise the query to one suitable for the solver. The NbE procedure is itself non-trivial to support lifting of uninterpreted functions and if-then-elses. The NbE procedure has been formalised in Agda.

Unfortunately the talk wasn't under the best of circumstances. I am in Boston for the conference, but tested positive for COVID yesterday morning. I had to give the talk from my hotel room 10 floors above the conference, which I think is possibly the worst way to give a talk.

Simple semantics for defaults in Catala

Mon, 16 Jan 2023 00:00:00 +0000

Catala is a programming language for the law.

As described in an ICFP 2021 paper by Merigoux et al., Catala has multiple features designed for making it easy to write code that implements legal texts.

The feature I want to talk about here is Catala's support for default reasoning.

Default reasoning involves rules like “X is the case, except when Y, Z, ...”. Apparently, such patterns are common in the law.

The ICFP paper points out that default reasoning is analogous to exceptions in normal programming languages. The paper describes how to translate default reasoning into exceptions.

I think that a good way to see how it works is in terms of a simple “exceptions monad”.

Default Calculus

The authors define the “default calculus”, which is the λ-calculus extended with constants for undefinedness and conflict and a default operator.

The default operator looks like this:

〈e1, ..., en | j :- c〉

where the intended meaning is “if j is true then evaluate to c and if j is undefined or false then evaluate to undefined, unless any of e1 to en are defined, in which case if exactly one ei is defined then evaluate to ei, if more than one is defined then evaluate to conflict”.

The Catala ICFP paper defines an operational semantics for the default operator. The authors give a translation into a λ-calculus with exceptions. They have proved the translation correct using F star.

Catala values

I think a more perspicious way of looking at how the default operator works is to think of it in terms of a simple monad.

I'll use OCaml syntax.

A “Catala value” is either Conflict, Undefined or an actual Value:

type 'a catala =
  | Conflict
  | Undefined
  | Value of 'a

This definition is basically the Maybe/option/exceptions monad with two exception values.

There is an “information” ordering on Catala values, with Undefined at the bottom, Conflict at the top, and the Values in between.

I'll not use the monad structure much, but we can decompose Catala's default operator into three simpler operators on Catala values.

The first is a conflict-adverse combination operator, which can be used to combine two Catala values into one:

let (<+>) x y = match x, y with
  | Undefined, x | x, Undefined -> x
  | Conflict, _ | _, Conflict -> Conflict
  | Value _, Value _ -> Conflict

Undefined always loses, Conflict always wins, and two attempts at a value (even if they are equal) results in conflict.

This is a commutative monoid with Undefined as the unit, and it is monotone in both arguments.

It'll be useful to be able to combine a list of Catala values using <+>:

let combine_list xs =
  List.fold_left (<+>) Undefined xs

The guard operator is a sort of gate that evaluates to its second argument if the first is true, and undefined otherwise (unless there was a conflict).

let guard x y = match x with
  | Conflict -> Conflict
  | Value true -> y
  | Value false
  | Undefined -> Undefined

This is also monotone in both arguments.

Finally, the unless operator acts like a backwards try-catch. unless x y defaults to x unless y is defined or is in conflict. Operationally, it tries y first, and if it throws the Undefined exception x is used instead.

let unless x y = match y with
  | Conflict -> Conflict
  | Undefined -> x
  | Value y -> Value y

This is not monotone in its second argument.

With these operators, we can recover Catala's default operator. It defaults to a guarded value, unless an exception is defined. In the event of multiple potential definitions, conflict is signaled.

let default value ~justification ~exceptions =
  unless (guard justification value) (combine_list exceptions)

I think this gives a simpler account of Catala's default operator than the ICFP paper.

We can see that the non-monotone behaviour of the default operator arises from the unless operator, which you might expect from a try-catch like operator that rescues definition from undefinedness.

The unless operator is a kind of “try this if that didn't work” operator, like exception handling and ordered choice in PEGs. I reckon that a semantics of data types with negation would be helpful in understanding such operators by recording “that didn't work” as a negation.

Some questions.

Why is it a Conflict to combine two values that are equal? I think that Catala thinks of Conflict as a mistake in the code or in the drafting of the law. But if there is a conflict that has no effect, then is that bad? What if a Catala value was a set of possible outcomes?
This formulation must be easier to reason about in a theorem prover than the ICFP paper's operational semantics or exception-based implementation.
What is the connection to Default Logic? The Catala paper states that the default operator is based on prioritised default logic, but doesn't elaborate on the formal relationship. Catala is focused on definitions, and rules are treated as a boolean-valued definitions. But default logic only deals in rules?
The Catala implementation seems to have a “without exceptions” translation but it seems more complicated than what I've written above. What is the connection?

Data types with Negation

Sun, 15 Jan 2023 00:00:00 +0000

Back in April and June last year, at MSFP and at TYPES, I gave talks about some work I've done on trying to understand what it would mean to allow negation in data type definitions.

The gist of the talk is: what if we could define a predicate for even numbers like this in Agda (or Coq or Idris or Lean):

data Even : Nat → Set where
  zero : Even 0
  suc  : ∀ {n} → not (Even n) → Even (suc n)

So 0 is even, and suc n is even if n isn't.

But if we get this to work, then what would this declaration mean?

data Liar : Set where
  liar : not Liar → Liar

I think that data types with negation will be useful for describing the sequences of decisions taken by back tracking algorithms. For example, a PEG parser's ordered choice A / B means “try A, and if that doesn't work try B”. We need some kind of negation to capture the meaning of “if that doesn't work”.

The solution I have at the moment is a proof relevant version of the well founded semantics of logic programs with negation, which is related to the stable model semantics used in Answer Set Programming. The key step is to replace the 3-valued logic used in the well founded semantics with Chu spaces.

I'm still working out the best way to present the theory, and what reasoning principles these data types have, but these slides are the best presentation I have so far.

Slides and Video for “Resource Constrained Programming with Full Dependent Types”

Mon, 23 Nov 2020 00:00:00 +0000

Last Friday, I gave a talk (virtually) at the CCS Colloquim at Augusta University, hosted by Harley Eades III. The title was “Resource Constrained Programming with Full Dependent Types” and the abstract was:

I will talk about a system that combines Dependent Types and Linear Types. As an application of this system, I will show how to transport Martin Hofmann’s LFPL and Amortised Resource analysis systems for linear and polynomial time computation to full dependent types. This results in a system where unconstrained computations are permitted at the type level, but only polynominal time computations are permitted at the term level. The combined system now allows one to explore the world of propositions whose proofs are not only constructive, but also of restricted complexity.

The talk was recorded, and the video is on YouTube. The slides:

Quantitative Typing with Non-idempotent Intersection Types

Thu, 13 Aug 2020 00:00:00 +0000

This post contains a development of an Agda proof that, for the Call-by-Name (CBN) λ-calculus, the number of steps needed to reduce a term to weak head normal form (WHNF) in the Krivine Abstract Machine (KAM) is equal to the size of the term's typing derivation in a non-idempotent intersection type system.

The inverse also holds: if a term reduces to WHNF in the KAM, then it has a typing derivation, the size of which is equal to the number of steps it takes to reduce the term. Therefore, typability characterises termination, quantitatively.

The main result manifests as a “quantitative” subject reduction lemma: not only is typing preserved by each reduction step, but the size of the typing derivation goes down by exactly one in each step.

Non-idempotent intersection types can also be seen as a syntactic presentation of a denotational semantics of untyped λ-calculus in the category Rel of sets and relations for a particular reflexive object. The fact that subject reduction (and expansion) have a quantitative aspectyields a proof of adequacy for this semantics that doesn't rely on logical relations (I've haven't proved this in this file though).

The quantitative nature of non-idempotent intersection types was originally observed for head normalisation by Daniel de Carvalho in Execution Time of lambda-Terms via Denotational Semantics and Intersection Types.

module non-idempotent-intersection-types where

Imports

First we need some things from the Agda standard library:

open import Data.Nat using (ℕ; zero; suc; _+_)
open import Data.Nat.Properties using (+-assoc; +-comm; +-identityʳ; suc-injective)
open import Data.Fin using (Fin; zero; suc; _≟_)
open import Data.Product using (Σ-syntax; _×_; _,_; proj₁; proj₂)
open import Data.List using (List; []; _∷_; _++_; [_])
import Relation.Binary.PropositionalEquality as Eq
open Eq using (_≡_; refl; trans; cong; subst; sym; _≢_; cong₂)
open Eq.≡-Reasoning using (begin_; _≡⟨⟩_; step-≡; _∎)

Untyped λ-calculus

The syntax of the programming language we will look at is the same as the usual untyped λ-calculus. I've opted for an intrinsically well-scoped de Bruijn representation:

data term : ℕ → Set where
  `_  : ∀ {n} → Fin n → term n
  ƛ   : ∀ {n} → term (suc n) → term n
  _·_ : ∀ {n} → term n → term n → term n

infixl 20 _·_
infix 40 `_

Krivine Abstract Machine

The standard way to give an operational semantics to the untyped λ-calculus is to give a small step reduction relation where the main rule is β-reduction:

  (ƛ t) · t' ---> t [ t' ]

However, our goal here is to measure the complexity of programs via typing and counting the number of β reductions is not a very convincing way of doing this. Substitution does a non-constant amount of work, depending on how big t is and how often the 0th variable is used in it.

Instead, we want to use a semantics that performs a constant amount of work at each step. One way to do this is to use an abstract machine to interpret programs. Abstract machines break the complex process of β-reduction down into smaller steps, so they are more convincing ways to measure complexity.

The abstract machine we shall use here is the Krivine Abstract Machine (KAM), which implements reduction of λ terms to weak head normal form.

Rather than using substitution, the KAM uses closures: pairs of a term and an environment that assigns a closure to each variable in the term. As this description suggests, closures and environments are mutually defined:

mutual
  data env : ℕ → Set where
    nil  : env zero
    _,-_ : ∀ {n} → env n → clo → env (suc n)

  clo = Σ[ n ∈ ℕ ] env n × term n

Note that environments are built by extending to the right.

The second ingredient for the KAM are stacks of closures. Stacks are used to remember the context within a program we are currently executing. Compare to the congruence rule used in the definition of CBN small step operational semantics:

       M  -->  M'
 ---------------------
  M · N  -->  M' · N

In the rule, the context is the N term, and in general we might have to go under several Ns to get to the part we are currently executing. The KAM uses a stack of closures to represent this:

data stk : Set where
  emp  : stk
  _-,_ : clo → stk → stk

infixr 10 _-,_

Stacks are built by extending to the left.

A configuration of the KAM is a pair of a closure, the current focus of the computation, and a stack representing the evaluation context. We use the special notation ⟨_⋆_⟩ for configurations.

record configuration : Set where
  constructor ⟨_⋆_⟩
  field
    closure : clo
    stack   : stk

infix  20 ⟨_⋆_⟩

The reduction semantics of the KAM is given by the following four rules, which are driven by the shape of the term in the first component of the configuration:

data _⇒_ : configuration → configuration → Set where
  grab : ∀ {n η c π}   → ⟨ suc n , η ,- c , ` zero    ⋆ π ⟩      ⇒ ⟨ c                      ⋆ π ⟩
  skip : ∀ {n η c i π} → ⟨ suc n , η ,- c , ` (suc i) ⋆ π ⟩      ⇒ ⟨ n ,     η        , ` i ⋆ π ⟩
  push : ∀ {n η t s π} → ⟨     n , η      , t · s     ⋆ π ⟩      ⇒ ⟨ n ,     η        , t   ⋆ (n , η , s) -, π ⟩
  pop  : ∀ {n η t c π} → ⟨     n , η      , ƛ t       ⋆ c -, π ⟩ ⇒ ⟨ suc n , (η ,- c) , t   ⋆ π ⟩

infixr 10 _⇒_

The rules grab and skip handle the execution of variables by incrementally looking them up in the current environment. One could also define a KAM that has a single rule for variable lookup (looking arbitrarily deep into the environment in a single step), but we chose to break it down into individual steps here.

The rule push executes an application t · s by pushing the argument s on to the stack (accompanied by its environment). Execution will then carry on in the function position (compare to the congruence rule given above for a small step operational semantics).

Finally, the rule pop executes a λ abstraction ƛ t by popping the current argument from the stack and adding it to the current environment. Combined with the first two rules, this performs a delayed β-reduction.

To start the KAM, we need to take programs that we want to execute and turn them into configurations:

init : term 0 → configuration
init t = ⟨ 0 , nil , t ⋆ emp ⟩

Due to the fact that we have defined our KAM using intrinsically scoped syntax, it is not possible for the KAM to get stuck by not having a variable in scope. The only way the machine can get stuck is by ending up with a λ-abstraction paired with an empty stack. In this case we can say that the original term has been reduced to its weak head normal form (WHNF). It also possible that a term never reduces to a final configuration at all -- not all terms finish reducing under the Call-by-Name semantics. To identify which terms reduce, we make an inductive definition that also records how many steps it takes for a term to reduce:

data reduces : ℕ → configuration → Set where
  stop : ∀ {n} η t → reduces 0 ⟨ n , η , ƛ t ⋆ emp ⟩
  step : ∀ {n p q} → p ⇒ q → reduces n q → reduces (suc n) p

So if we have a value of type reduces n p, then we know that a configuration p reduces to a WHNF in n steps.

Since the selection of rules the KAM is driven by the shape of the term being executed, execution in the KAM is deterministic:

deterministic : ∀ {p q q'} → p ⇒ q → p ⇒ q' → q ≡ q'
deterministic grab grab = refl
deterministic skip skip = refl
deterministic push push = refl
deterministic pop  pop  = refl

Consequently, any configuration that reduces does so in a unique number of steps:

reduces-det : ∀ {p m n} → reduces m p → reduces n p → m ≡ n
reduces-det (stop η t)   (stop .η .t) = refl
reduces-det (step s1 r1) (step s2 r2) with deterministic s1 s2
... | refl = cong suc (reduces-det r1 r2)

The Non-Idempotent Type Assignment System

Non-idempotent intersection types assign types to terms according to their precise runtime behaviour. Usually type systems assign types that are over approximations of the possible behaviours of a program. Intersection types (whether idempotent or not) assign types that track precisely the behaviour a program will have (in a particular context). Indeed, as we will see below, the system we present here precisely tracks the termination behaviour of programs. This precision does make typechecking undecidable.

Types, Observations, Behaviours

The types of our system are defined as:

data type : Set where
  ⋆    : type
  _↦_ : List type → type → type

infixr 30 _↦_

We can interpret the types of the system as kinds of “observations” or “behaviour descriptions” that we can make of a program. Per the definition of type, we have two kinds of observation in our system:

The observation ⋆ means that we can observe that the term will reduce to a value (a λ abstraction), but gives us no further information. To observe that a complete program reduces to WHNF, we will seek typings that type the whole program with the observation ⋆.
The observation σ ↦ τ pairs a list of observations σ with a single observation τ. Making this observation means that the term will reduce to a function that when applied to an argument that exhibits all the behaviours in σ, exhibits the observation τ.

For example, the observation [⋆] ↦ ⋆ is one that we can assign to the identity function: we can observe that the output is a value, as long as it can make the observation that the input is a value.

The lists of types σ are the intersection types of our system. They indicate that a particular term must have all of the behaviours listed, so it must be in the intersection of all the possible behaviours: σ = τ₁ ∧ τ₂ ∧ ... ∧ τₙ. Idempotent intersection type systems disregard multiple occurrences of the same behaviour (i.e. τ ∧ τ = τ) in an intersection: they only track whether a behaviour is present or not. Non-idempotent intersection type systems, in contrast, track the number of occurrences of each behaviour. In turn, this tracks the number of times a function uses its arguments. It is this quantitative aspect that will allow us to use non-idempotent intersection types to measure the reduction complexity of programs.

To see how the rejection of idempotency allows tracking of number of uses, consider the following two terms (written using explicit names):

  λf. λx. f x

and

  λf. λx. f (f x)

When applied to the identity function λx .x and any value λz.t, these terms both reduce to a value. We can see this by the types that the system we define below will assign to them. The first one gets the following type:

  ⊢ λf. λx. f x ⦂ [[⋆] ↦ ⋆] ↦ [⋆] ↦ ⋆

In words, this says "if the first argument yields values when given values, and the second argument is a value, then the result is a value". The second term is assigned a similar type, but here we can see the quantitative nature of the system coming in to play:

  ⊢ λf. λx. f (f x) ⦂ [[⋆] ↦ ⋆, [⋆] ↦ ⋆] ↦ [⋆] ↦ ⋆

Notice that the type now states that the first argument will be used twice, because two behaviours appear in the first intersection. This precisely tracks the structure of the term, which does indeed use f twice. If we assumed idempotency, then the two types would be equivalent, and we would be tracking the qualitative behaviour, but not the quantitative behaviour.

Typing contexts

To assign types to open terms, we need typing contexts. Typing contexts assign to each free variable position a list of possible behaviours. Typing contexts grow to the right.

data ctx : ℕ → Set where
  nil : ctx zero
  _,-_ : ∀ {n} → ctx n → List type → ctx (suc n)

We need the following two combinators for typing contexts. The first, empty, is the typing context where every variable is assumed to have no behaviours. This is used to type terms that do not use any of the variables in scope.

empty : ∀{n} → ctx n
empty {zero}  = nil
empty {suc n} = empty {n} ,- []

The _+++_ combinator “zips” two typing contexts to combine their possible behaviours. We will use this to create contexts that combine the observations on contexts required by two the two subterms in an application term.

_+++_ : ∀{n} → ctx n → ctx n → ctx n
nil        +++ nil       = nil
(Γ₁ ,- σ₁) +++ (Γ₂ ,- σ₂) = (Γ₁ +++ Γ₂) ,- (σ₁ ++ σ₂)

Non-idempotent intersection type systems are sensitive to the number of occurrences of each behaviour in an intersection, but not to their order. We will model this by requiring equivalence only up to permutation at various places. We define a permutation between two lists via sequences of swappings:

module permutations {X : Set} where
  data _⋈_ : List X -> List X -> Set where
    nil     : [] ⋈ []
    skip    : ∀ {x l₁ l₂}  -> l₁ ⋈ l₂ -> (x ∷ l₁) ⋈ (x ∷ l₂)
    swap    : ∀ {x y l}    ->           (x ∷ y ∷ l) ⋈ (y ∷ x ∷ l)
    ⋈-trans : ∀ {l₁ l₂ l₃} -> l₁ ⋈ l₂ -> l₂ ⋈ l₃ -> l₁ ⋈ l₃

  ⋈-refl : ∀ {l} -> l ⋈ l
  ⋈-refl {[]}    = nil
  ⋈-refl {x ∷ l} = skip ⋈-refl
open permutations

We use permutations to define when two typing contexts are pointwise permutations of each other. I.e., when two contexts have the same number of variables, but the observations at each variable are permuted.

data _⋈ctx_ : ∀ {n} → ctx n → ctx n → Set where
  nil : nil ⋈ctx nil
  _,-_ : ∀ {n}{Γ₁ Γ₂ : ctx n}{σ₁ σ₂} → Γ₁ ⋈ctx Γ₂ → σ₁ ⋈ σ₂ → (Γ₁ ,- σ₁) ⋈ctx (Γ₂ ,- σ₂)

⋈ctx-refl : ∀ {n}{Γ : ctx n} → Γ ⋈ctx Γ
⋈ctx-refl {Γ = nil}    = nil
⋈ctx-refl {Γ = Γ ,- σ} = ⋈ctx-refl ,- ⋈-refl

Type Assignment

We can now define the rules of our system. First, we define what it means for a variable (represented as an Agda value of type Fin n) to be well typed in a context:

data _⊢v_⦂_ : ∀ {n} → ctx n → Fin n → type → Set where
  zero : ∀ {n τ} → (empty {n} ,- [ τ ]) ⊢v zero ⦂ τ
  suc  : ∀ {n τ i}{Γ : ctx n} → Γ ⊢v i ⦂ τ → (Γ ,- []) ⊢v (suc i) ⦂ τ

The key point here is that the context for a variable must require no behaviours for any variable that is not the one being selected, and must require exactly one behaviour for the one being selected. Thus one use a variable corresponds to one observation.

The typing rules for terms are mutually defined with a rule for typing a single term against an intersection type. I will explain the rules after the definition.

mutual
  data _⊢_⦂_ : ∀ {n} → ctx n → term n → type → Set where
    var : ∀ {n τ i}{Γ : ctx n} →
          Γ ⊢v i ⦂ τ →
          Γ ⊢ ` i ⦂ τ
    lam : ∀ {n σ τ t}{Γ : ctx n} →
          (Γ ,- σ) ⊢ t ⦂ τ →
          Γ ⊢ ƛ t ⦂ (σ ↦ τ)
    app : ∀ {n σ τ s t}{Γ Γ₁ Γ₂ : ctx n} →
          Γ₁ ⊢ s ⦂ (σ ↦ τ) →
          Γ₂ ⊢ t ⦂' σ →
          (Γ₁ +++ Γ₂) ⋈ctx Γ →
          Γ ⊢ (s · t) ⦂ τ
    lam⋆ : ∀ {n t} →
          empty {n} ⊢ ƛ t ⦂ ⋆

  data _⊢_⦂'_ : ∀ {n} → ctx n → term n → List type → Set where
    nil : ∀ {n}{t : term n} →
          empty ⊢ t ⦂' []
    _,~_∷_ : ∀ {n t σ τ}{Γ₁ Γ₂ Γ : ctx n} →
          Γ₁ ⊢ t ⦂ τ →
          (Γ₁ +++ Γ₂) ⋈ctx Γ →
          Γ₂ ⊢ t ⦂' σ →
          Γ ⊢ t ⦂' (τ ∷ σ)

The rule var uses the variable typing judgement to assign types to terms that are variables. The rule lam states that a λ abstraction can be assigned the type σ ↦ τ if the body can be assigned the type τ under the assumption that the 0th variable has the intersection type σ. The app rule states that if the term s can be observed to act like a function mapping intersections of behaviours σ to behaviours τ, and the term t exhibits all the behaviours in σ (using the auxiliary judgement _⊢_⦂'_), then the whole application term exhibits the behaviour τ. Note that the two terms are typed in separate contexts that are combined -- so the uses of variables in s and t are tracked separately. The rule lam⋆ is used to assign the observation that λ abstractions are values (⋆ means that the term is a value) to any λ abstraction.

The auxiliary judgement Γ ⊢ t ⦂' σ is used to ensure that a single term t can exhibit all the possible behaviours in σ. Note that each individual behaviour in σ must be observable under a separate typing context. All the typing contexts are summed up (up to permutation), to give the complete requirements on the context.

This completes the definition of the type assignment system. We demonstrate it using the two terms from above. These are defined as follows:

--- λf. λx. f x
example₁ : term 0
example₁ = ƛ (ƛ (` suc zero · ` zero))

-- λf. λx. f (f x)
example₂ : term 0
example₂ = ƛ (ƛ (` suc zero · (` suc zero · ` zero)))

To give typing derivations for these, we will save a bit of typing by using the following derived rule for application when we apply a function that will only use its argument once:

app₁ : ∀ {n Γ₁ Γ₂ τ₁ τ₂}{s t : term n} →
      Γ₁ ⊢ s ⦂ [ τ₁ ] ↦ τ₂ →
      Γ₂ ⊢ t ⦂ τ₁ →
      (Γ₁ +++ (Γ₂ +++ empty)) ⊢ s · t ⦂ τ₂
app₁ s-ok t-ok = app s-ok (t-ok ,~ ⋈ctx-refl ∷ nil) ⋈ctx-refl

With this derived rule to help us, the first example can be assigned the type we gave above like so:

typing₁ : nil ⊢ example₁ ⦂ [ [ ⋆ ] ↦ ⋆ ] ↦ [ ⋆ ] ↦ ⋆
typing₁ = lam (lam (app₁ (var (suc zero)) (var zero)))

And the second one like so:

typing₂ : nil ⊢ example₂ ⦂ ([ ⋆ ] ↦ ⋆ ∷ [ ⋆ ] ↦ ⋆ ∷ []) ↦ [ ⋆ ] ↦ ⋆
typing₂ = lam (lam (app₁ (var (suc zero)) (app₁ (var (suc zero)) (var zero))))

In the second one we see how the type reveals that the term uses its first argument twice.

Measuring the sizes of derivations

Our main result below is that typing is preserved by execution quantitatively. Therefore, we need to be able to compute the sizes of typing derivations. This is done by the following three functions:

v-size : ∀ {n}{Γ : ctx n}{t τ} → Γ ⊢v t ⦂ τ → ℕ
v-size zero    = 1
v-size (suc d) = 1 + v-size d

mutual
  size : ∀ {n}{Γ : ctx n}{t τ} → Γ ⊢ t ⦂ τ → ℕ
  size (var x)      = v-size x
  size (lam d)      = 1 + size d
  size (app d₁ d₂ _) = 1 + size d₁ + size' d₂
  size lam⋆         = 0

  size' : ∀ {n}{Γ : ctx n}{t σ} → Γ ⊢ t ⦂' σ → ℕ
  size' nil           = 0
  size' (d ,~ _ ∷ ds) = size d + size' ds

We have been a little selective in what we have counted. We have added 1 for every construct that can cause the KAM to make a step: whether a variable lookup (zero or suc), an application, or a λ abstraction that we expect to have some interesting behaviour. In particular a use of lam⋆ is counted as 0 because it represents values that will not be inspected any further, and so will not cause the KAM to do any work.

The size' function adds up all the derivations used in checking an intersection type. This means that we accurately account for all the possible ways that a single term will be used.

Typing KAM configurations.

We now extend the non-idempotent intersection typing discipline from terms to configurations of the KAM. This is required so that we can state our main lemmas.

Closures are typed by ensuring that the term contained in them is well typed for some context, and the associated environment is contains closures well typed according to that context. Because contexts contain intersection types, we need an auxiliary judgement that checks that a closure is well typed for all types in an intersection. These three judgements (closures, environments, closures with intersection types) need to be mutually defined:

mutual
  data ⊢c_⦂_ : (c : clo) (τ : type) → Set where
    t-clo : ∀ {n t τ}{η : env n}{Γ : ctx n} →
       Γ ⊢ t ⦂ τ →
       ⊢e η ⦂ Γ →
       ⊢c (n , η , t) ⦂ τ

  data ⊢e_⦂_ : ∀ {n} → env n → ctx n → Set where
    nil : ⊢e nil ⦂ nil
    _,-_ : ∀{n}{η : env n}{Γ : ctx n}{c}{σ} →
           ⊢e η ⦂ Γ →
           ⊢c c ⦂' σ →
           ⊢e (η ,- c) ⦂ (Γ ,- σ)

  data ⊢c_⦂'_ : (c : clo) (σ : List type) → Set where
    nil : ∀ {c} →
           ⊢c c ⦂' []
    _∷_ : ∀ {c τ σ} →
           ⊢c c ⦂ τ →
           ⊢c c ⦂' σ →
           ⊢c c ⦂' (τ ∷ σ)

We also extend the measurement of sizes of derivations to well typed closures and environments. Notice that these derivations do not contribute themselves to the size: we only sum up the sizes of the contained term typing derivations.

mutual
  c-size : ∀ {c τ} → ⊢c c ⦂ τ → ℕ
  c-size (t-clo t e) = size t + e-size e

  e-size : ∀ {n}{η : env n}{Γ} → ⊢e η ⦂ Γ → ℕ
  e-size nil = 0
  e-size (d ,- c) = e-size d + c'-size c

  c'-size : ∀ {c σ} → ⊢c c ⦂' σ → ℕ
  c'-size nil = 0
  c'-size (c ∷ c') = c-size c + c'-size c'

Stacks are typed with two types τ₁ ⊸ τ₂, meaning that if the observation τ₁ can be made of the computation happening under the stack, then the observation τ₂ can be observed at the "top level":

data ⊢s_⦂_⊸_ : (π : stk) (τ₁ τ₂ : type) → Set where
  emp  : ∀ {τ} →
         ⊢s emp ⦂ τ ⊸ τ
  _-,_ : ∀ {c π σ τ₁ τ₂} →
         ⊢c c ⦂' σ →
         ⊢s π ⦂ τ₁ ⊸ τ₂ →
         ⊢s (c -, π) ⦂ (σ ↦ τ₁) ⊸ τ₂

Again, we measure the size of a stack typing derivation by adding up the sizes of the typing derivations contained within:

s-size : ∀ {π τ₁ τ₂} → ⊢s π ⦂ τ₁ ⊸ τ₂ → ℕ
s-size emp = 0
s-size (c -, π) = c'-size c + s-size π

Well typed KAM configurations are constructed by pairing closures that can generate a behaviour τ₁ with stacks that can turn that behaviour into a behaviour τ at the top level. The size of a configuration is measured by summing up the sizes of the typing derivations included in the closure and the stack.

data ⊢p_⦂_ : configuration → type → Set where
  cfg : ∀ {c π τ₁ τ} →
         ⊢c c ⦂ τ₁ →
         ⊢s π ⦂ τ₁ ⊸ τ →
         ⊢p ⟨ c ⋆ π ⟩ ⦂ τ

cfg-size : ∀ {p τ} → ⊢p p ⦂ τ → ℕ
cfg-size (cfg c π) = c-size c + s-size π

We can now prove that the initial configuration of the KAM for a given program is well typed if the program is:

init-typed : ∀{t τ} → nil ⊢ t ⦂ τ → ⊢p init t ⦂ τ
init-typed d = cfg (t-clo d nil) emp

The size of this initial typing derivation is also equal to the size of the original typing derivation:

init-typed-size : ∀{t τ} → (d : nil ⊢ t ⦂ τ) → cfg-size (init-typed d) ≡ size d
init-typed-size{τ = τ} d =
  begin
     size d + e-size nil + s-size{τ₁ = τ} emp
   ≡⟨⟩
     size d + 0 + 0
   ≡⟨ +-identityʳ (size d + 0) ⟩
     size d + 0
   ≡⟨ +-identityʳ (size d) ⟩
     size d
   ∎

Quantitative Subject Reduction

We now connect the KAM semantics to the non-idempotent intersection type system. This will take the form of three key lemmas:

Subject Reduction (a.k.a preservation), which says that if a configuration of the KAM is well typed, and steps to another configuration, then that configuration is also well-typed. The twist here is that this lemma is also quantitative: the size of the typing derivation after one step is exactly one smaller than it was before. This will allow us to prove that all typable terms reduce.
Subject Expansion, the converse of subject reduction. If a configuration is well typed and was the result of a step of the KAM, then the predecessor was also well typed. Subject expansion does not usually hold for type systems, but is characteristic of intersection type systems due to the way they describe what a program must do, instead of what it may do. Subject expansion allows us to prove a completeness result: if a program reduces to WHNF, then we can step it backwards to recover a typing derivation for the initial state.
Progress: if a configuration is well typed, and its derivation has non-zero size, then the KAM can take a step. Thus we can interpret the size of a derivation as the amount of computation that a program has left to perform.

Some lemmas

Before we can prove subject reduction, we will need some auxiliary lemmas about the type system. These mostly say that typing is preserved under permutation, splitting, or joining of intersection types. We also prove that these preservations are size preserving, anticipating the quantitative subject reduction lemma in the next section.

First, we need that if a closure c can be well typed for (i.e., exhibits the behaviour of) all the types in σ₁ then it is well typed for any permutation of σ₁. Also, this does not affect the size of the typing derivation. We prove this by induction on the permutation applied:

clo-permute : ∀ {c σ₁ σ₂} → ⊢c c ⦂' σ₁ → σ₂ ⋈ σ₁ → ⊢c c ⦂' σ₂
clo-permute d            nil            = d
clo-permute (x ∷ d)      (skip p)       = x ∷ clo-permute d p
clo-permute (x ∷ x₁ ∷ d) swap           = x₁ ∷ x ∷ d
clo-permute d            (⋈-trans p p₁) = clo-permute (clo-permute d p₁) p

clo-permute-size : ∀ {c σ₁ σ₂} → (d : ⊢c c ⦂' σ₁) → (p : σ₂ ⋈ σ₁) → c'-size d ≡ c'-size (clo-permute d p)
clo-permute-size d              nil            = refl
clo-permute-size (c ∷ d)        (skip p)       = cong (λ □ → c-size c + □) (clo-permute-size d p)
clo-permute-size (x ∷ (x₁ ∷ d)) swap           = trans (sym (+-assoc (c-size x) (c-size x₁) (c'-size d))) (trans (cong (λ □ → □ + c'-size d) (+-comm (c-size x) (c-size x₁))) (+-assoc (c-size x₁) (c-size x) (c'-size d)))
clo-permute-size d              (⋈-trans p p₁) = trans (clo-permute-size d p₁) (clo-permute-size (clo-permute d p₁) p)

Preservation of closure typing under permutation extends to preservation of environment typing under pointwise permutation of typing contexts, and this is also size preserving:

env-permute : ∀ {n}{η : env n}{Γ₁ Γ₂ : ctx n} →
             ⊢e η ⦂ Γ₁ →
             Γ₂ ⋈ctx Γ₁ →
             ⊢e η ⦂ Γ₂
env-permute nil      nil       = nil
env-permute (d ,- x) (p ,- x₁) = env-permute d p ,- clo-permute x x₁

env-permute-size : ∀ {n}{η : env n}{Γ₁ Γ₂ : ctx n} →
             (d : ⊢e η ⦂ Γ₁) → (p : Γ₂ ⋈ctx Γ₁) →
             e-size d ≡ e-size (env-permute d p)
env-permute-size nil      nil       = refl
env-permute-size (d ,- x) (p ,- x₁) = cong₂ _+_ (env-permute-size d p) (clo-permute-size x x₁)

If a closure c exhibits all the behaviours in the big intersection σ₁ ++ σ₂, then it exhibits them separately. Moreover, the sizes of the two separate typing derivations sums to the original:

clo-split : ∀ {c σ₁ σ₂} → ⊢c c ⦂' (σ₁ ++ σ₂) → ⊢c c ⦂' σ₁ × ⊢c c ⦂' σ₂
clo-split {σ₁ = []}     d        = nil , d
clo-split {σ₁ = x ∷ σ₁} (x₁ ∷ d) = let d₁ , d₂ = clo-split d in x₁ ∷ d₁ , d₂

clo-split-size : ∀ {c σ₁ σ₂} → (d : ⊢c c ⦂' (σ₁ ++ σ₂)) → c'-size d ≡ c'-size (clo-split {σ₁ = σ₁} d .proj₁) + c'-size (clo-split {σ₁ = σ₁} d .proj₂)
clo-split-size {σ₁ = []}     d        = refl
clo-split-size {σ₁ = x ∷ σ₁} (x₁ ∷ d) = trans (cong (λ □ → c-size x₁ + □) (clo-split-size {σ₁ = σ₁} d)) (sym (+-assoc (c-size x₁) _ _))

As a consequence of clo-split, we can prove that if an environment exhibits behaviours in two contexts combined, then it exhibits them separately. We will use this to distribute welltypedness of environments between the two parts of an application term in the push rule of the KAM.

env-split : ∀ {n} {η : env n} {Γ₁ Γ₂} → ⊢e η ⦂ (Γ₁ +++ Γ₂) → ⊢e η ⦂ Γ₁ × ⊢e η ⦂ Γ₂
env-split {n = zero}  {nil}    {nil}      {nil}     nil       = nil , nil
env-split {n = suc n} {η ,- x} {Γ₁ ,- x₁} {Γ₂ ,- x₂} (d ,- x₃) = (env-split d .proj₁ ,- clo-split x₃ .proj₁) , (env-split d .proj₂ ,- clo-split x₃ .proj₂)

The splitting of typing derivations by env-split preserves sizes, but to prove this we first need a lemma about addition:

interchange : ∀ a b c d → (a + b) + (c + d) ≡ (a + c) + (b + d)
interchange a b c d =
  begin
    (a + b) + (c + d)
  ≡⟨ +-assoc a b (c + d) ⟩
    a + (b + (c + d))
  ≡⟨ cong (λ □ → a + □) (sym (+-assoc b c d)) ⟩
    a + ((b + c) + d)
  ≡⟨ cong (λ □ → a + (□ + d)) (+-comm b c) ⟩
    a + ((c + b) + d)
  ≡⟨ cong (λ □ → a + □) (+-assoc c b d) ⟩
    a + (c + (b + d))
  ≡⟨ sym (+-assoc a c (b + d)) ⟩
    (a + c) + (b + d)
  ∎

With this lemma, we can prove that splitting an environment typing derivation is size preserving:

env-split-size : ∀ {n} {η : env n} {Γ₁ Γ₂} → (d : ⊢e η ⦂ (Γ₁ +++ Γ₂)) → e-size d ≡ e-size (env-split d .proj₁) + e-size (env-split d .proj₂)
env-split-size {n = zero}  {nil}    {nil}      {nil}     nil      = refl
env-split-size {n = suc n} {η ,- x} {Γ₁ ,- σ₁} {Γ₂ ,- σ₂} (d ,- c) =
  begin
     e-size d + c'-size c
   ≡⟨ cong₂ _+_ (env-split-size d) (clo-split-size {x}{σ₁} c) ⟩
     (e-size (env-split d .proj₁) + e-size (env-split d .proj₂)) + (c'-size (clo-split {x}{σ₁} c .proj₁) + c'-size (clo-split {x}{σ₁} c .proj₂))
   ≡⟨ interchange (e-size (env-split d .proj₁)) (e-size (env-split d .proj₂)) (c'-size (clo-split {x}{σ₁} c .proj₁)) (c'-size (clo-split {x}{σ₁} c .proj₂)) ⟩
     e-size (env-split d .proj₁) + c'-size (clo-split {x}{σ₁} c .proj₁) + (e-size (env-split d .proj₂) + c'-size (clo-split {x}{σ₁} c .proj₂))
   ∎

We also have nullary version of clo-split and env-split: whenever a closure has is typed to have no behaviours, or an environment consists of closures that have no required behaviours, then the sizes of their derivations is zero:

c'-empty : ∀ {c} → (d : ⊢c c ⦂' []) → c'-size d ≡ 0
c'-empty nil = refl

e-empty : ∀ {n}{η : env n} → (d : ⊢e η ⦂ empty) → e-size d ≡ 0
e-empty nil = refl
e-empty (d ,- c) = begin
                      e-size d + c'-size c
                   ≡⟨ cong₂ _+_ (e-empty d) (c'-empty c) ⟩
                      0
                   ∎

The final piece we need before proving subject reduction is the following derived typing rule for constructing typings of closures against intersections from typings of terms against intersections, assuming an appropriately typed environment. This uses the env-split and env-permute lemmas.

mk-clo : ∀ {n}{η : env n}{Γ : ctx n}{s : term n}{σ} →
         ⊢e η ⦂ Γ →
         Γ ⊢ s ⦂' σ →
         ⊢c n , η , s ⦂' σ
mk-clo {σ = []} η-ok nil  = nil
mk-clo {σ = τ ∷ σ} η-ok (x ,~ p ∷ s-ok) =
  t-clo x (env-split (env-permute η-ok p) .proj₁) ∷ mk-clo (env-split (env-permute η-ok p) .proj₂) s-ok

This derived rule is again size preserving, using the fact that env-split and env-permute are size preserving:

mk-clo-size : ∀ {n}{η : env n}{Γ : ctx n}{s : term n}{σ} →
              (η-ok : ⊢e η ⦂ Γ) →
              (s-ok : Γ ⊢ s ⦂' σ) →
              size' s-ok + e-size η-ok ≡ c'-size (mk-clo η-ok s-ok)
mk-clo-size {σ = []} η-ok nil = e-empty η-ok
mk-clo-size {σ = τ ∷ σ} η-ok (x ,~ p ∷ s-ok) =
  begin
     size x + size' s-ok + e-size η-ok
   ≡⟨ cong (λ □ → size x + size' s-ok + □) (env-permute-size η-ok p) ⟩
     size x + size' s-ok + e-size (env-permute η-ok p)
   ≡⟨ cong (λ □ → size x + size' s-ok + □) (env-split-size (env-permute η-ok p)) ⟩
     size x + size' s-ok + (e-size (env-split (env-permute η-ok p) .proj₁) + e-size (env-split (env-permute η-ok p) .proj₂))
   ≡⟨ interchange (size x) (size' s-ok) (e-size (env-split (env-permute η-ok p) .proj₁)) (e-size (env-split (env-permute η-ok p) .proj₂)) ⟩
     size x + e-size (env-split (env-permute η-ok p) .proj₁) + (size' s-ok + e-size (env-split (env-permute η-ok p) .proj₂))
   ≡⟨ cong (λ □ → size x + e-size (env-split (env-permute η-ok p) .proj₁) + □) (mk-clo-size (env-split (env-permute η-ok p) .proj₂) s-ok) ⟩
     size x + e-size (env-split (env-permute η-ok p) .proj₁) + c'-size (mk-clo (env-split (env-permute η-ok p) .proj₂) s-ok)
   ∎

Subject Reduction

Using the lemmas in the previous section, we can now prove the subject reduction property for the non-idempotent intersection type system. The proof consists of taking each possible KAM step in turn, pattern matching on the possible starting configurations' typing derivations, and constructing the corresponding typing derivation for the ending configuration:

subject-reduction : ∀ {p q τ} → p ⇒ q → ⊢p p ⦂ τ → ⊢p q ⦂ τ
subject-reduction grab (cfg (t-clo (var zero)    (_    ,- (c ∷ nil))) π-ok)        = cfg c π-ok
subject-reduction skip (cfg (t-clo (var (suc x)) (η-ok ,- _))         π-ok)        = cfg (t-clo (var x) η-ok) π-ok
subject-reduction push (cfg (t-clo (app t s p)   η-ok)                π-ok)        =
  cfg (t-clo t (env-split (env-permute η-ok p) .proj₁)) (mk-clo (env-split (env-permute η-ok p) .proj₂) s -, π-ok)
subject-reduction pop  (cfg (t-clo (lam t)       η-ok)                (c -, π-ok)) = cfg (t-clo t (η-ok ,- c)) π-ok

In most of the cases, the final typing derivation is constructed from parts of the initial one. In the application case, we need to use env-split to distribute the typing derivation of the environment (η-ok) between the closures for the function and its argument.

We now prove that subject reduction is quantitative: the typing derivation generated by subject reduction is always of size one smaller than the initial one. This is a consequence of the fact that most of the operations we perform on typing derivations in he subject reduction proof are size preserving, except for the fact that we remove the top level term former, which accounts for the decrementing by one.

sr-size : ∀ {p q τ} →
          (step : p ⇒ q) →
          (d : ⊢p p ⦂ τ) →
          cfg-size d ≡ suc (cfg-size (subject-reduction step d))
sr-size grab (cfg (t-clo (var zero)    (η-emp ,- (c ∷ nil))) π-ok)        =
  begin
     suc (e-size η-emp + (c-size c + 0) + s-size π-ok)
   ≡⟨ cong (λ □ → suc (□ + (c-size c + 0) + s-size π-ok)) (e-empty η-emp) ⟩
     suc (0 + (c-size c + 0) + s-size π-ok)
   ≡⟨⟩
     suc ((c-size c + 0) + s-size π-ok)
   ≡⟨ cong (λ □ → suc (□ + s-size π-ok)) (+-identityʳ (c-size c)) ⟩
     suc (c-size c + s-size π-ok)
   ∎
sr-size skip (cfg (t-clo (var (suc x)) (η-ok  ,- c'))        π-ok)        =
  begin
     suc (v-size x + (e-size η-ok + c'-size c') + s-size π-ok)
   ≡⟨ cong (λ □ → suc (v-size x + (e-size η-ok + □) + s-size π-ok)) (c'-empty c') ⟩
     suc (v-size x + (e-size η-ok + 0) + s-size π-ok)
   ≡⟨ cong (λ □ → suc (v-size x + □ + s-size π-ok)) (+-identityʳ (e-size η-ok)) ⟩
     suc (v-size x + e-size η-ok + s-size π-ok)
   ∎
sr-size push (cfg (t-clo (app t s p)   η-ok)                 π-ok)        =
  begin
     suc (size t + size' s + e-size η-ok + s-size π-ok)
  ≡⟨ cong (λ □ → suc (size t + size' s + □ + s-size π-ok)) (env-permute-size η-ok p) ⟩
     suc (size t + size' s + e-size (env-permute η-ok p) + s-size π-ok)
  ≡⟨ cong (λ □ → suc (size t + size' s + □ + s-size π-ok)) (env-split-size (env-permute η-ok p)) ⟩
     suc (size t + size' s + (e-size (env-split (env-permute η-ok p) .proj₁) + e-size (env-split (env-permute η-ok p) .proj₂)) + s-size π-ok)
  ≡⟨ cong (λ □ → suc (□ + s-size π-ok)) (interchange (size t) (size' s) (e-size (env-split (env-permute η-ok p) .proj₁)) (e-size (env-split (env-permute η-ok p) .proj₂))) ⟩
     suc (size t + e-size (env-split (env-permute η-ok p) .proj₁) + (size' s + e-size (env-split (env-permute η-ok p) .proj₂)) + s-size π-ok)
  ≡⟨ cong (λ □ → suc (size t + e-size (env-split (env-permute η-ok p) .proj₁) + □ + s-size π-ok)) (mk-clo-size (env-split (env-permute η-ok p) .proj₂) s) ⟩
     suc (size t + e-size (env-split (env-permute η-ok p) .proj₁) + c'-size (mk-clo (env-split (env-permute η-ok p) .proj₂) s) + s-size π-ok)
  ≡⟨ cong (λ □ → suc □) (+-assoc (size t + e-size (env-split (env-permute η-ok p) .proj₁)) (c'-size (mk-clo (env-split (env-permute η-ok p) .proj₂) s)) (s-size π-ok)) ⟩
     suc (size t + e-size (env-split (env-permute η-ok p) .proj₁) + (c'-size (mk-clo (env-split (env-permute η-ok p) .proj₂) s) + s-size π-ok))
  ∎
sr-size pop  (cfg (t-clo (lam t)       η-ok)                 (c -, π-ok)) =
  begin
     suc (size t + e-size η-ok + (c'-size c + s-size π-ok))
  ≡⟨ cong (λ □ → suc □) (+-assoc (size t) (e-size η-ok) (c'-size c + s-size π-ok)) ⟩
     suc (size t + (e-size η-ok + (c'-size c + s-size π-ok)))
  ≡⟨ cong (λ □ → suc (size t + □)) (sym (+-assoc (e-size η-ok) (c'-size c) (s-size π-ok))) ⟩
     suc (size t + ((e-size η-ok + c'-size c) + s-size π-ok))
  ≡⟨ cong suc (sym (+-assoc (size t) (e-size η-ok + c'-size c) (s-size π-ok))) ⟩
     suc (size t + (e-size η-ok + c'-size c) + s-size π-ok)
  ∎

When we combine these two lemmas with the progress lemma we prove below, we will be able to prove by induction on the size of the typing derivation that the execution of the KAM terminates. This is the content of the soundness lemma below.

Subject expansion

A feature of intersection type systems is that they often enjoy subject expansion as well as subject reduction. The type information is a precise enough description of a term's behaviour that we are always able to reconstruct typing derivations backwards through execution.

To prove subject expansion we will need the inverse to the closure and environment splitting lemmas we proved for subject reduction:

clo-join : ∀ {c σ₁ σ₂} → ⊢c c ⦂' σ₁ → ⊢c c ⦂' σ₂ → ⊢c c ⦂' (σ₁ ++ σ₂)
clo-join {σ₁ = []}     nil      c₂ = c₂
clo-join {σ₁ = τ ∷ σ₁} (x ∷ c₁) c₂ = x ∷ clo-join c₁ c₂

env-join : ∀ {n} {η : env n} {Γ₁ Γ₂} → ⊢e η ⦂ Γ₁ → ⊢e η ⦂ Γ₂ → ⊢e η ⦂ (Γ₁ +++ Γ₂)
env-join {n = zero}  {nil}    {nil}      {nil}     nil       nil        = nil
env-join {n = suc n} {η ,- c} {Γ₁ ,- σ₁} {Γ₂ ,- σ₂} (e₁ ,- c₁) (e₂ ,- c₂) = env-join e₁ e₂ ,- clo-join c₁ c₂

We also need the fact that every environment is well typed if the types we use make no demands:

env-empty : ∀ {n}{η : env n} → ⊢e η ⦂ empty
env-empty {η = nil}    = nil
env-empty {η = η ,- x} = env-empty ,- nil

Together, these allow us to prove that the derived rule for building closures typed with intersection types (mkclo) is invertible:

unmk-clo : ∀ {n}{η : env n}{s : term n}{σ} →
         ⊢c n , η , s ⦂' σ →
         Σ[ Γ ∈ ctx n ] (⊢e η ⦂ Γ × Γ ⊢ s ⦂' σ)
unmk-clo {σ = []} nil = empty , env-empty , nil
unmk-clo {σ = τ ∷ σ} (t-clo {Γ = Γ} s-τ η-ok ∷ d) with unmk-clo d
... | Γ' , η-ok' , s-σ = (Γ +++ Γ') , env-join η-ok η-ok' , (s-τ ,~ ⋈ctx-refl ∷ s-σ)

The proof of subject expansion is now has a similar structure to the proof of subject reduction:

subject-expansion : ∀ {p q τ} → p ⇒ q → ⊢p q ⦂ τ → ⊢p p ⦂ τ
subject-expansion grab (cfg c-ok π-ok) = cfg (t-clo (var zero) (env-empty ,- (c-ok ∷ nil))) π-ok
subject-expansion skip (cfg (t-clo (var x) x₂) x₁) = cfg (t-clo (var (suc x)) (x₂ ,- nil)) x₁
subject-expansion push (cfg (t-clo {Γ = Γ} t-ok η-ok) (c-ok -, π-ok)) with unmk-clo c-ok
... | Γ' , η-ok' , s-ok = cfg (t-clo (app t-ok s-ok ⋈ctx-refl) (env-join η-ok η-ok')) π-ok
subject-expansion pop  (cfg (t-clo t (x₂ ,- x₃)) x₁) = cfg (t-clo (lam t) x₂) (x₃ -, x₁)

Progress

As is usual progress only holds for complete programs. In our case, that means configurations that are typed with the observation ⋆, meaning that we expect the configuration to terminate with a value. Our formulation of progress is quantitative, so rather than examining the configuration p to see whether it is a value or not, we rely on measuring the amount of “potential” left in the configuration by measuring its size. If this is non-zero, then the KAM will make progress.

progress : ∀ {p} → (d : ⊢p p ⦂ ⋆) → cfg-size d ≢ 0 → Σ[ q ∈ configuration ] p ⇒ q
progress (cfg (t-clo (var zero)    (η-emp ,- (c ∷ _))) π-ok)        _ = _ , grab
progress (cfg (t-clo (var (suc x)) (η-ok ,- nil))      π-ok)        _ = _ , skip
progress (cfg (t-clo (lam t)       η-ok)               (c -, π-ok)) _ = _ , pop
progress (cfg (t-clo (app s t p)   η-ok)               π-ok)        _ = _ , push
progress (cfg (t-clo lam⋆          η-ok)               emp)         r rewrite e-empty η-ok with r refl
... | ()

Quantitative Soundness and Completeness

Putting all our results together, we can now prove that non-idempotent intersection types are a sound and complete characterisation of termination in the CBN λ-calculus, and moreover the size of the typing derivation is exactly the number of steps taken to terminate.

We first do soundness: if a closed term t has a typing derivation d that shows that ⋆ is a possible observation of t, then t will reduce in the KAM to WHNF in size d steps:

soundness : ∀ {t} → (d : nil ⊢ t ⦂ ⋆) → reduces (size d) ⟨ 0 , nil , t ⋆ emp ⟩
soundness {t} d = subst (λ □ → reduces □ ⟨ 0 , nil , t ⋆ emp ⟩)
                        (init-typed-size d)
                        (run (init-typed d) (cfg-size (init-typed d)) refl)
  where
    helper : ∀ {k n} → n ≡ suc k → n ≢ 0
    helper refl ()

    final : ∀ {p} → (d : ⊢p p ⦂ ⋆) → cfg-size d ≡ 0 → reduces 0 p
    final (cfg (t-clo (var zero) _)    π-ok) ()
    final (cfg (t-clo (var (suc _)) _) π-ok) ()
    final {⟨ n , η , ƛ t ⋆ emp ⟩} (cfg (t-clo lam⋆ x₁) emp) e = stop η t

    run : ∀ {p} → (d : ⊢p p ⦂ ⋆) → ∀ k → cfg-size d ≡ k → reduces k p
    run d zero    e = final d e
    run d (suc k) e with progress d (helper e)
    ... | q , s = step s (run (subject-reduction s d) k (suc-injective (trans (sym (sr-size s d)) e)))

The bulk of the work in this lemma is performed by the run function which uses induction on the size of the typing derivation to repeatedly call progress to get the next step, subject-reduction to preserve typability, and sr-size to show that the size of the derivation goes down.

The converse of soundness is completeness: every term that reduces to WHNF has a typing derivation. This is proved by induction on the reduction sequence: in the base case all WHNFs are typable by the lam⋆ rule, and in the step case by using subject expansion. We also know by soundness and the determinism of length of reduction sequences that the computed typing derivation has the same size as the length of the reduction.

completeness : ∀ {t n} → reduces n ⟨ 0 , nil , t ⋆ emp ⟩ → Σ[ d ∈ nil ⊢ t ⦂ ⋆ ] (size d ≡ n)
completeness {t} r = d , reduces-det (soundness d) r
  where
   config-typed : ∀ {p k} → reduces k p → ⊢p p ⦂ ⋆
   config-typed (stop η t) = cfg (t-clo lam⋆ env-empty) emp
   config-typed (step s r) = subject-expansion s (config-typed r)

   extract : ∀ {t} → ⊢p ⟨ 0 , nil , t ⋆ emp ⟩ ⦂ ⋆ → nil ⊢ t ⦂ ⋆
   extract (cfg (t-clo x nil) emp) = x

   d : nil ⊢ t ⦂ ⋆
   d = extract (config-typed r)

Inspecting the proof of subject expansion, we can also see that the computed typing derivation never uses a non-identity permutation, indicating that we might be able to use a variant of non-idempotent intersection types to predict the ordering of usage, as well as quantity of usage.

A possible application of this type system would be to proving that optimisations actually optimise. If we have an optimisation that replaces a term M with a term N, then if for all typings Γ ⊢ M ⦂ τ we can get a typing Γ ⊢ N ⦂ τ that is smaller, then any context which uses M can replace it with N, preserving meaning, and take fewer steps to terminate.

Slides for “Resource Constrained Programming with Full Dependent Types”

Fri, 24 Jan 2020 00:00:00 +0000

Yesterday, I gave a talk entitled “Resource Constrained Programming with Full Dependent Types” at the IRIF Proofs, Programs and systems seminar in Paris. Many thanks to Paul-André Melliès for inviting me to give a talk.

Abstract:

I will talk about a system that combines Dependent Types and Linear Types. As an application of this system, I will show how to transport Martin Hofmann's LFPL and Amortised Resource analysis systems for resource constrained computing to full dependent types. This results in a theory where unconstrained computations are allowed at the type level, but only polynomial time computations at the term level. The combined system allows one to explore the world of propositions whose proofs are not only constructive, but also of restricted complexity.

Slides:

Off the Beaten Track 2017: Call for Talk Proposals

Mon, 10 Oct 2016 00:00:00 +0000

This year I am the programme chair for Off the Beaten Track 2017! This will be held on 21st January 2017, co-located with POPL 2017 in Paris, France.

Background

Programming language researchers have the principles, tools, algorithms and abstractions to solve all kinds of problems, in all areas of computer science. However, identifying and evaluating new problems, particularly those that lie outside the typical core PL problems we all know and love, can be a significant challenge. This workshop’s goal is to identify and discuss problems that do not often show up in our top conferences, but where programming language research can make a substantial impact. We hope fora like this will increase the diversity of problems that are studied by PL researchers and thus increase our community’s impact on the world.

While many workshops associated with POPL have become more like mini-conferences themselves, this is an anti-goal for OBT. The workshop will be informal and structured to encourage discussion. We are at least as interested in problems as in solutions.

Scope

A good submission is one that outlines a new problem or an interesting, underrepresented problem domain. Good submissions may also remind the PL community of problems that were once in vogue but have not recently been seen in top PL conferences. Good submissions do not need to propose complete or even partial solutions, though there should be some reason to believe that programming languages researchers have the tools necessary to search for solutions in the area at hand. Submissions that seem likely to stimulate discussion about the direction of programming language research are encouraged.

Use your imagination. It's hard to imagine how a paper that discusses programming languages could be considered out of scope. If in doubt, ask the program chair.

Previous OBTs

2017 marks the sixth year of OBT and its co-location with POPL. The previous five workshops were:

OBT 2016, St. Petersburg, USA
OBT 2015, Mumbai, India
OBT 2014, San Diego, USA
OBT 2013, Rome, Italy
OBT 2012, Philadelphia, USA

Important Dates

10th November 2016: Submission deadline
8th December 2016: Notification
(18th December 2016: POPL early registration)
21st January 2017: Workshop

Submission

Please submit your talk proposal via EasyChair:

https://easychair.org/conferences/?conf=obt2017

All submissions should be in PDF format, two pages or less, in at least 10pt font, printable on A4 and on US Letter paper. Authors are welcome to include links to multimedia content such as YouTube videos or online demos. Reviewers may or may not view linked documents; it is up to authors to convince the reviewers to do so.

For each accepted submission, one of the authors will give a talk at the workshop. The length of the talk will depend on the submissions received and how the program committee decides to assemble the program.

Reviewing of submissions will be very light. Authors should not expect a detailed analysis of their submission by the program committee. Accepted submissions will be posted as is on this web site. By submitting a document, you agree that if it is accepted, it may be posted and you agree that one of the co-authors will attend the workshop and give a talk there. There will be no revision process and no formal publication.

Organisers

General chair:

Lindsey Kuper, Intel Labs, USA

Programme chair:

Robert Atkey, University of Strathclyde, UK

Programme committee:

Ekaterina Komendantskaya, Heriot-Watt University, UK
Chris Martens, North Carolina State University, USA
Tomas Petricek, University of Cambridge, UK
Wren Romano, Google Inc., USA
Mary Sheeran, Chalmers University of Technology, Sweden
KC Sivaramakrishnan, University of Cambridge, UK
Wouter Swierstra, Utrecht University, Netherlands

Authenticated Data Structures, as a Library, for Free!

Tue, 12 Apr 2016 00:00:00 +0000

Let's assume that you're querying to some database stored in the cloud (i.e., on someone else’s computer). Being of a sceptical mind, you worry whether or not the answers you get back are from the database you expect. Or is the cloud lying to you?

Authenticated Data Structures (ADSs) are a proposed solution to this problem. When the server sends back its answers, it also sends back a “proof” that the answer came from the database it claims. You, the client, verify this proof. If the proof doesn't verify, then you’ve got evidence that the server was lying. If the proof does verify, then there is a guarantee that the server’s response is legitimate (usually up to the possibility of a hash collision).

This all seems great, but doesn’t address the question of how anyone might build an ADS, and, crucially, prove that it has the security of unforgable (up to hash collisions) verification. A brute-force way to implement an ADS is for the client to retain a copy of the database, and to check the server’s results against the copy. But then what would be the point of the server?

Merkle Trees are the original example of an ADS. Merkle trees solve the problem of needing a complete copy of the database by having the client store a hash of the database. The server’s proof is then verified against the hash. But what if we want a data structure that isn’t trees? Will we have to implement our own ADS? How will we know that we have done it correctly? Implementing a completely new ADS has three problems. We need: to invent a way to do the authentication, a proof that authentication has been done correctly, and a proof that we have correctly implemented it.

Andrew Miller, Michael Hicks, Jonathan Katz, and Elaine Shi have a solution to this meta-problem in their POPL2014 paper Authenticated Data Structures, Generically, by describing a new language “Lambda-Auth” for implementing correct-by-construction ADSs.

Miller et al. describe a programming language with special constructs for writing ADSs. Once you’ve written an implementation of your data structure, you annotate the implementation with ‘authentication’ markers that indicate points in the structure that act as authentication checkpoints. On the client (verifier) side, each checkpoint becomes a hash code representing the real data, which will be checked against the proof sent by the server. On the server (prover) side, each checkpoint is an indicator of where to generate a piece of proof. The key insight of Miller et al. is that the client and server run the same code, just with two different interpretations of what the authentication checkpoints mean. They are then able to prove, for all programs, that the server and client sides will always agree, and that proofs of authentication are unforgable (up to hash collisions). The authors wrote a PL Enthusiast blog post with a gentle introduction, and a second part with more detail.

In this post, I'll show that in a language with sufficiently powerful abstraction facilities (in this case OCaml), it is possible to implement Miller et al.’s solution as a library within the language, with no need to create a new language, or to alter an existing language’s implementation.

Moreover, although I won’t give any details in this blog post, I claim that the correctness proof given by Miller et al. will be an instance of parametricity for higher-kinded types. I’ll give a few sketchy details at the end.

Authenticated Data Structures as a Library

To get started, I need to isolate a description of the bits and pieces we need to be able to write authenticated data structures, taking Miller et al.’s paper as a guide. OCaml provides module signatures as a handy way of collecting together requirements for the existence of certain types and functions. I'll call the types and functions we need for making ADSs an AUTHENTIKIT: a kit for implementing authenticated data structures:

module type AUTHENTIKIT = sig

The first thing is to postulate the existence of the type constructor auth that represents the type of authenticated values. This is the OCaml rendering of the “•” type constructor used in Miller et al.’s language for representing authenticated values. Note that I haven’t committed to any implementation of this type: it is left completely abstract in the AUTHENTIKIT interface:

  type 'a auth

We’ll need is a way of describing abstract authenticated computations: computations that are either generating proofs or verifying proofs. In Miller et al.’s setup, authenticated computations are built in to the language. However, OCaml comes with with its own fixed notion of computation, so we have to use a monad to layer our own notions on top:

  type 'a authenticated_computation
  val return : 'a -> 'a authenticated_computation
  val (>>=)  : 'a authenticated_computation ->
               ('a -> 'b authenticated_computation) ->
               'b authenticated_computation

The third thing I’ll need is a way of proving that the types of data that we want to authenticate are “authenticatable”. This will essentially mean that they are serialisable to a string representation in way that is suitable for proof construction and verification. The requirement to be serialisable is hidden in Miller et al.’s formalism because they assume that all data in the language is serialisable. This is a somewhat fishy assumption (how do you serialise and deserialise a function without breaking abstraction boundaries?), so here I’m being a bit more formal, and requiring the programmer to build evidence of serialisability using the following combinators tucked away in a submodule of any AUTHENTIKIT implementation:

  module Authenticatable : sig
    type 'a evidence
    val auth   : 'a auth evidence
    val pair   : 'a evidence -> 'b evidence -> ('a * 'b) evidence
    val sum    : 'a evidence -> 'b evidence ->
                      [`left of 'a | `right of 'b] evidence
    val string : string evidence
    val int    : int evidence
  end

Finally, I postulate the existence of the auth and unauth functions as Miller et al. do. The difference here is that I have explicitly requested evidence that the data type involved is authenticatable. Also, the unauth function returns a computation in our authenticated computation monad, indicating that this is where some of the work to get an authenticated data structure to work will happen.

  val auth   : 'a Authenticatable.evidence -> 'a -> 'a auth

  val unauth : 'a Authenticatable.evidence ->
               'a auth ->
               'a authenticated_computation
end

An Autheticated Data Structure: Merkle Trees

Before I describe the prover and verifier implementations of the AUTHENTIKIT module type, I’ll give an example of an authenticated data structure implementation. I’ll do a basic Merkle tree, with the following interface:

module type MERKLE =
  functor (A : AUTHENTIKIT) -> sig
    open A

    type path = [`L | `R] list
    type tree = [`left of string | `right of tree * tree] auth

    val make_leaf : string -> tree
    val make_branch : tree -> tree -> tree
    val retrieve :
	  path -> tree -> string option authenticated_computation
    val update :
	  path -> string -> tree -> tree option authenticated_computation
  end

The first thing to note about this module signature is that its implementations are parameterised by implementations of AUTHENTIKIT. This will be characteristic of any authenticated data structure implementation we implement in this style: we need the freedom to instantiate the implementation with alternative AUTHENTIKITs to get the prover and verifier sides of the system.

The second thing to note is that the data structure interface is a very lightly annotated version of a completely boring binary tree interface. We have types for trees and paths, and operations to construct trees, query trees, and update trees. The only differences from a normal interface is that the tree type is annotated with an extra auth type constructor, indicating how the tree is augmented with authentication information, and the authenticated_computation type constructors on the retrieve and update operations, indicating that they are authenticated operations.

The implementation is likewise a straightforward implementation of a binary tree in OCaml, with a couple of extra annotations for the authentication. The Merkle module signature states that implementations are parameterised by AUTHENTIKITs, so we declare the Merkle implementation like so:

module Merkle : MERKLE =
  functor (A : AUTHENTIKIT) -> struct
    open A

The open A introduces all the members of the AUTHENTIKIT into scope, so we don't have to qualify any names. The first thing to do is to fulfil the promise of definitions for the types path and tree that we gave in the interface:

    type path = [`L|`R] list
    type tree = [`left of string | `right of tree * tree ] auth

Next, we construct some evidence that the body of the tree type is authenticatable, so we will be able to use auth and unauth on trees.

    let tree :
     [`left of string | `right of tree * tree] Authenticatable.evidence =
      Authenticatable.(sum string (pair auth auth))

The make_leaf and make_branch functions build leaves and branches using the appropriate constructors.

    let make_leaf s =
      auth tree (`left s)

    let make_branch l r =
      auth tree (`right (l,r))

To query our authenticated data structure, we have two functions retrieve and update. First, the implementation of retrieve which takes a path and a tree and returns the data identified by that path, if it exists.

    let rec retrieve path t =
      unauth tree t >>= fun t ->
      match path, t with
        | [],       `left s      -> return (Some s)
        | `L::path, `right (l,r) -> retrieve path l
        | `R::path, `right (l,r) -> retrieve path r
        | _,        _            -> return None

As I mentioned above, this is really nothing more than a standard binary tree search implementation, obfuscated by the need to unwrap the authenticated input, and the additional >>= and returns needed to track the authenticated computation. The update implemention is similar in its similarlity to a standard binary tree update function:

    let rec update path v t =
      unauth tree t >>= fun t ->
      match path, t with
        | [],       `left _      ->
           return (Some (make_leaf v))
        | `L::path, `right (l,r) ->
           (update path v l >>= function
             | None    -> return None
             | Some l' -> return (Some (make_branch l' r)))
        | `R::path, `right (l,r) ->
           (update path v r >>= function
             | None    -> return None
             | Some r' -> return (Some (make_branch l r')))
      | _ ->
         return None

Finally, an end to complete the definition of Merkle:

end

Just to emphasise again, this implementation is essentially the same as a standard binary tree implementation written in OCaml. In the code, we didn’t have to mention anything to do with authenticated data structures, except for placing auth and unauth in the correct places. And even if we get in a muddle doing that, the OCaml type checker will helpfully inform us where we’ve made a mistake. One wrinkle is that I've had to give up writing in “direct style” and had to write in monadic style.

The Prover

The Merkle module is not yet ready for use. To use it we need an implementation of the AUTHENTIKIT interface. The different implementations of AUTHENTIKIT will correspond to the different semantics in Miller et al.’s presentation: the prover and the verifier.

The first implementation of AUTHENTIKIT I'll give is the prover.

Proofs

The prover constructs proofs, which I'll represent as lists of JSON values, using the Ezjsonm library.

type proof = Ezjsonm.value list

We will also need to compute cryptographic hashes of JSON values, which I'll do using the Cryptokit library. I'm using the SHA1 algorithm here, but obviously any secure hashing algorithm would work.

let hash_json =
  let hash_algo = Cryptokit.Hash.sha1 () in
  fun json_value ->
    Cryptokit.hash_string hash_algo (Ezjsonm.to_string (`A json_value))

Prover Implementation

The module Prover implements the AUTHENTIKIT signature, plus the additional knowledge that a) authenticated computations generate proofs as well as values, and b) that we have a get_hash function that returns the hashed representation of any authenticated value. The prover’s interface is captured by its module signature:

module Prover : sig
  include AUTHENTIKIT with type 'a authenticated_computation = proof * 'a
  val get_hash : 'a auth -> string
end = struct

First, we implement the prover’s view of authenticated values by defining its implementation of the auth type. The prover sees an authenticated value as a pair of an underlying value x and a hash of x’s representation:

  type 'a auth = 'a * string

  let get_hash (a,h) = h

It will be up to the rest of the functions in this module to maintain the invariant that the second half of the pair will always be the hash code of the serialised representation of the first half.

Authenticated computations on the prover side are represented using a specialisation of the standard Writer monad, which collects a proof (list of JSON values) as a side-effect. This implementation is quite slow, because I have used lists, but could be speeded up by using a better data structure.

  type 'a authenticated_computation =
    proof * 'a

  let return a =
    ([], a)

  let (>>=) (prf,a) f =
    let (prf',b) = f a in
    (prf @ prf',b)

The prover's view of authenticatable values are ones for which it is possible to serialise them to JSON. We represent the evidence that such a thing is possible as the existence of a function from values to JSON values.

  module Authenticatable = struct
    type 'a evidence = 'a -> Ezjsonm.value

    let auth (a,h) =
      `String h

    let pair a_serialiser b_serialiser (a,b) =
      `A [a_serialiser a; b_serialiser b]

    let sum a_serialiser b_serialiser = function
      | `left a  -> `A [`String "left"; a_serialiser a]
      | `right b -> `A [`String "right"; b_serialiser b]

    let string s = `String s
    let int i = `String (string_of_int i)
  end

In the auth case, we only serialise the hash code, not the underlying value. This ensures that the prover does not end up sending whole data structures back to the client. In the int case, I'm serialising OCaml’s int values as strings, to avoid complications arising from JSON’s use of floating point representations for numbers.

Now the auth and unauth functions. Creation of authenticated values means pairing the value with its hashed serialised representation:

  let auth serialiser a =
    (a, hash_json (serialiser a)))

Extracting the underlying value from an authenticated value has the “side effect” of producing a step in the proof, which is the JSON representation of the value we expect to see:

  let unauth serialiser (a, h) =
    ([serialiser a], a)

Finally, we complete the definition of Prover with an end:

end

Trying out the Prover

We get a prover-side Merkle tree implementation by instantiating the Merkle module with the Prover implementation of AUTHENTIKIT:

module Merkle_Prover = Merkle (Prover)

Let’s make a little prover-side tree, in the OCaml REPL:

# let tree =
    Merkle_Prover.(make_branch
                     (make_branch (make_leaf "a") (make_leaf "b"))
                     (make_branch (make_leaf "c") (make_leaf "d")));;
val tree : Merkle_Prover.tree = <abstr>

The returned value is abstract, because it is really a pair of the underlying data and it hash code. Using the prover’s get_hash function, we can get the hash code of the tree, which is what the verifier will use to authenticate the prover’s actions:

# let code = Prover.get_hash tree;;
val code : string = ".z\129w\199J\224\\\254\220\bo\246W\158\243S\029\177\190"

We can also ask the prover to do queries on the tree. For example, if we ask for the value at position “left, left”, it returns the result “a”, and a proof that the verifier will be able to use to check that this result actually came from the tree with the hash code above.

# let proof, result = Merkle_Prover.retrieve [`L;`L] tree;;
val proof : proof =
    [`A [ `String "right"
        ; `A [ `String "?\250m&,\251\129\031\r\252QJ\001\141|d}\242\016l"
             ; `String "i?B\230p\158D\201\248\145\000\1400p\224\018\023\219\1935"
             ]
        ]
    ; `A [ `String "right"
         ; `A [ `String "X\140\005\028\146\1891L\224\246\224\229\201\018o\b\187\163\240\160"
              ; `String "\223\231\194\230\1362=\157\187\226;?\143>\127\248\014;\201\254"
              ]
         ]
    ; `A [`String "left"; `String "a"]
    ]
val result : string option = Some "a"

This proof looks quite long, and in this case is several times the size of the original database. So it might look like Merkle trees might not save us anything. In general, however, the proof size is logarithmic in the size of the tree and so will be much smaller than sending the entire tree.

The Verifier

We’ve got a prover generating query responses and proofs, but this is a bit useless if we don’t have a verifier to check them.

Our verifier is another implementation of the AUTHENTIKIT signature, with the additional information that authenticated computations are now functions that consume proofs and return either a value (and possibly some left-over proof) or return failure. Also, we specify that, on the verifier side, an authenticated value is represented as just its hash code (represented as an OCaml string).

module Verifier : sig
  include AuthentiKit
    with type 'a authenticated_computation =
                   proof -> [ `Ok of proof * 'a | `ProofFailure ]
     and type 'a auth = string
end = struct

The basic idea behind the verifier is that, given a hash code and a proof, it checks the hash code against the proof, and then uses the hash code to rebuild the parts of the data structure that will be explored by the program.

We start by fulfilling our statement that the verifier’s view of authenticated values is as OCaml strings:

  type 'a auth = string

And that the verifier’s version of authenticated computations is as a “parser” of proofs. Note how the definitions here are very similar to the definitions used for parser combinators. In some sense, verification is a process of “parsing” the proof sequence supplied by the prover.

  type 'a authenticated_computation =
    json list -> [`Ok of proof * 'a | `ProofFailure]

  let return a =
    fun proof -> `Ok (proof, a)

  let (>>=) c f =
    fun prfs ->
      match c prfs with
        | `ProofFailure -> `ProofFailure
        | `Ok (prfs',a) -> f a prfs'

The verifier’s view of authenticable values is slightly more involved than the prover’s, because it needs to be able to serialise and deserialise values to and from JSON. There isn’t anything particularly special going on here, but we do have to be careful to ensure that this implementation uses the same format as the prover’s. (An obvious improvement is to make the Prover and Verifier implementations share an implementation of this sub-module.)

  module Authenticatable = struct
    type 'a evidence =
      { serialise   : 'a            -> Ezjsonm.value
      ; deserialise : Ezjsonm.value -> 'a option
      }

    let auth =
      let serialise h = `String h
      and deserialise = function
        | `String s -> Some s
        | _         -> None
      in
      { serialise; deserialise }

    let pair a_s b_s =
      let serialise (a,b) =
        `A [a_s.serialise a; b_s.serialise b]
      and deserialise = function
        | `A [x;y] ->
           (match a_s.deserialise x, b_s.deserialise y with
             | Some a, Some b -> Some (a,b)
             | _ -> None)
        | _ ->
           None
      in
      { serialise; deserialise }

    let sum a_s b_s =
      let serialise = function
        | `left a  -> `A [`String "left"; a_s.serialise a]
        | `right b -> `A [`String "right"; b_s.serialise b]
      and deserialise = function
        | `A [`String "left"; x] ->
           (match a_s.deserialise x with
             | Some a -> Some (`left a)
             | _ -> None)
        | `A [`String "right"; y] ->
           (match b_s.deserialise y with
             | Some b -> Some (`right b)
             | _ -> None)
        | _ ->
           None
      in
      { serialise; deserialise }

    let string =
      let serialise s = `String s
      and deserialise = function
        | `String s -> Some s
        | _         -> None
      in
      { serialise; deserialise }

    let int =
      let serialise i = `String (string_of_int i)
      and deserialise = function
        | `String i -> (try Some (int_of_string i) with Failure _ -> None)
        | _         -> None
      in
      { serialise; deserialise }
  end

Now we get to the crucial auth and unauth functions. Creation of authenticated values on the verifier side is nothing more than serialising them and computing their hash code:

  open Authenticatable

  let auth auth_evidence a =
    hash_json (auth_evidence.serialise a))

Finally, we get to the actual proof checking. The unauth function is supplied with a) a (de)serialiser for the type of value to be produced; b) a hash code for the expected value; and c) a proof which will be used to reconstitute the actual value. If we have run out of proof elements, then we fail immediately. Otherwise, we check that the hash code of the first item in the proof is the same as our required code. If so, we deserialise the proof item and return it with the remainder of the proof. If the hash codes do not match, then the verifier reports that proof checking has failed.

  let unauth auth_evidence h proof = match proof with
    | [] -> `ProofFailure
    | p::ps when hash_json p = h ->
       (match auth_evidence.deserialise p with
         | None   -> `ProofFailure
         | Some a -> `Ok (ps, a))
    | _ -> `ProofFailure

Finally, the Verifier ends with an end:

end

Trying out the Verifier

Above, we used the Merkle_Prover module to generate a small Merkle tree and run a query on it. We can now verify the prover’s execution of retrieve. Recall that we extracted a hash code representing the tree from the prover’s representation:

val code : string = ".z\129w\199J\224\\\254\220\bo\246W\158\243S\029\177\190"

Let’s assume that this code was conveyed to the client somehow, and the client trusts that it is an accurate representation of the tree it wants to query.

Now the client/verifier asks the server/prover to perform a query. The server sends back the result and a proof, which we computed above:

val proof : proof =
    [`A [ `String "right"
        ; `A [ `String "?\250m&,\251\129\031\r\252QJ\001\141|d}\242\016l"
             ; `String "i?B\230p\158D\201\248\145\000\1400p\224\018\023\219\1935"
             ]
        ]
    ; `A [ `String "right"
         ; `A [ `String "X\140\005\028\146\1891L\224\246\224\229\201\018o\b\187\163\240\160"
              ; `String "\223\231\194\230\1362=\157\187\226;?\143>\127\248\014;\201\254"
              ]
         ]
    ; `A [`String "left"; `String "a"]
    ]
val result : string option = Some "a"

We can now use Merkle_Verifier to verify the prover’s proof against code:

# Merkle_Verifier.retrieve [`L;`L] code proof;;
- : [ `Ok of proof * string option | `ProofFailure ] = `Ok ([], Some "a")

Attempting to trick the verifier

Let’s try simulating an attempt by the prover to trick the verifier by running the query against a different tree. First we create a tree with the same shape, but with different values in it:

let other_tree =
    Merkle_Prover.(make_branch
                     (make_branch (make_leaf "A") (make_leaf "B"))
                     (make_branch (make_leaf "C") (make_leaf "D")));;

Now we run the “left, left” query on this alternative tree, and we get back a result and a proof:

# let proof, result = Merkle_Prover.retrieve [`L;`L] other_tree;;
val proof : proof =
   [ `A [`String "right"
        ; `A [`String "Q\217G\000\246\238!\248\212\127\194\184\179>\017zW\0182("
             ; `String "0\239\238\002b4\172\145\127\143\002@-=g\179\197\022\154|"
             ]
        ]
   ; `A [`String "right"
        ; `A [`String "\157\134\234N\234QS\136\165\196\0038\133j\018uY\133\030\005"
             ; `String "\2272|\223M\230\241\132\167\029-\016\141\221\005nx\239\190\184"
             ]
        ]
   ; `A [`String "left"; `String "A"]
   ]
val result : bytes option = Some "A"

We send the result and proof back to the sceptical client, who verifies the proof against their hash code:

# Merkle_Verifier.retrieve [`L;`L] code proof;;
- : [ `Ok of proof * bytes option | `ProofFailure ] = `ProofFailure

It fails! The cheating server has been thwarted.

Correctness from Parametricity

Of course, the examples above don’t prove that the server/prover can never cheat. For their calculus, Miller et al. provide a proof that the prover and verifier implementations satisfy the following property, which I’ve stated informally here:

If the verifier accepts a proof (i.e., returns Ok) then either it came from the prover, or is the result of a hash collision.

If we assume that hash collisions are unlikely (Miller et al. make this assumption more formal), then we get the security property we want: up to the possibility of hash collision, we can spot cheating clouds.

I claim, but I’m not going to prove here, that this property is provable as an instance of the “Free Theorem” we get from the fact that the Merkle implementation is parameterised by AUTHENTIKIT implementations. The full proof involves F-ing Modules to translate the use of modules and functors into the higher-kinded calculus System Fω, my Relational Parametricity for Higher Kinds for the higher-kinded version of free theorems, and Katsumata’s A Semantic Formulation of TT-lifting and Logical Predicates for Computational Metalanguage to handle the authenticated computations monads.

Slides and notes for my OBT “Generalising Abstraction” talk

Tue, 26 Jan 2016 00:00:00 +0000

Here are the slides I used for my keynote talk for the afternoon at this year’s Off the Beaten Track workshop in St. Petersburg, Florida, USA. Many thanks to Lindsey Kuper for inviting me to give a talk and for organising it all.

The talk wasn’t recorded, but here is a collection of notes to go with the things I talked about:

The Abstract is ‘an Enemy’ is a paper by Alan F. Blackwell, Luke Church, and Thomas Green, written in response to Jeannette Wing’s highly influential Computational Thinking. Wing argues, amongst other things, that abstraction is a key feature of the set of techniques that computer scientists use when thinking about problems, and that people working in other disciplines could profitably make use of some of the abstraction tools developed by computer scientists. Blackwell et al. disagree with Wing’s emphasis on abstraction, noting that it can lead to system designs that de-emphasise and dehumanise users in favour of simple and tractable models designed for programmers. They also argue that abstraction can also lead to excessive literalism, conflation of the goals of a system with the goal of “the perfect abstraction”, and mathematical abstraction as a tool for theoreticians to hide behind when defending their theories. Admittedly, I put this slide up to get a cheap laugh from their abstract, but I do think the paper is worth reading in its own right.
The “Pedestrian or Equestrian” sign is from Stanley Park in Vancouver. I went over the equestrian side. The combination of the pine forest visuals and the water dripping from the ceiling in the room where OBT was held made for quite the Vancouverean experience.
Jean-Yves Girard’s Locus Solum is the paper that introduces Ludics, a logical formalism whose details persist in eluding my understanding. An interesting feature of the paper is the extensive glossary covering a large portion of logic, semantics, proof theory, and computer science in a unique way (Appendix A: A pure waste of paper).
Barbara Liskov and Stephen Zilles’ Programming with Abstract Data Types was the first paper to really nail down the idea of an abstract data type as a structured thing described through a set of operations, instead of the more negative definition in terms of information hiding. This lead to the design of the language CLU, led by Liskov, and to a great amount of work in algebraic specification. The Power of Abstraction is a excellent and informative talk by Liskov covering the history of how she invented abstract data types.
Types, Abstraction, and Parametric Polymorphism by John Reynolds is the founding paper of a semantic theory of abstraction, now called parametricity. Christopher Strachey’s original definition of parametricity was informally defined in terms of parametricity polymorphic functions being “uniformly defined” with respect to their type parameter. There are many ways to formalise this notion. We could strictly interpreting it as meaning there is a single implementation, ignoring the type parameter, as is done in PER models that interpret quantification over all types as intersection. Reynolds chooses to interpret parametricity as preservation of relations, generalising the invariance under homomorphism idea from algebra. This is formalised in the Abstraction Theorem which appears in this paper. As he notes, the Abstraction Theorem is closely related to the “fundamental”, or “basic”, lemma of Logical Relations, first used for the lambda-calculus by Gordon Plotkin for proving definability results. I believe the origin of logical relations goes back to Robin Gandy, who used them (or at least logical relations that are equivalence relations) to ensure extensionality in models of calculi with higher-order functions.
John Harrison’s Without Loss of Generality is a genuinely practical paper for reducing the effort involved in proving geometric theorems in an interactive theorem prover. Harrison takes inspiration from the way that mathematicians reduce problems to simpler ones by exploiting symmetry. The idea being that “without loss of generality” one can consider a special case that soundly represents all of the instances of the general case. The paper presents an extension to the HOL Light theorem prover that pattern matches on goals containing geometric statements and reduces them to simpler goals using symmetry.
The quote “A rose by any other name would smell as sweet” is from Romeo and Julliet by William Shakespeare. According to Wikipedia, this play about a catastrophic lack of invariance under renaming was written some time between 1591 and 1594.
Andy Pitts’ work on Nominal Sets is collected in his book Nominal Sets: Names and Symmetry in Computer Science. The quote on the slide is taken from the introductory notes Nominal Techniques. The essence of nominal sets is that everything ought to be invariant under the swapping of names; it is only the relationships between names that matter.
The reflexive graph model for System F is originally due to Edmund Robinson and Giuseppe Rosolini, who showed that it is possible to take existing denotational models of System F (i.e., typed lambda-calculus with “for all” types) and complete them to a relationally parametric ones by considering reflexive graphs whose sets of nodes and edges are denotations of types in the original model. I used this approach to give a relationally parametric model of System Fomega (i.e., System F plus type-level functions) in Relational Parametricity for Higher Kinds. A similar approach had been used by Ryu Hasegawa in Relational limits in general polymorphism. Neil Ghani, Patricia Johann, and myself used the reflexive graph approach to give a relationally parametric model of dependent types in A Relationally Parametric Model of Dependent Type Theory. The model I presented in the talk is essentially the same as the one in that paper, except that the reflexive graphs are presented in “indexed-style”, rather than “display-style”.
Dimension Types seem quite popular at the moment, with libraries popping up for many different languages that encode dimension types. However, most of these systems concentrate on the computer says “no” aspect. They are only interested in the type system preventing things, and womble on about Mars rockets going missing and so forth, like using a different programming language would have fixed that. Andrew Kennedy’s Relational Parametricity and Units of Measure is much more insightful because it shows how unit-correctness actually yields interesting results: including scale invariance results, type isomorphisms, and indefinability results. This was really the point of my talk: abstraction yields properties.
Philip Wadler dubbed some of the consequences of type abstraction that arise from Reynolds’ work “Theorems for Free!”, because the theorem is derived from the type instead of inspection of the program, which will often be more complex. This has led to many paper titles that end with “... for free!” when they use parametricity to prove something.
Distance-indexed types, where two objects are related if they are within a certain distance, are originally from Distance makes the Types Grow Stronger by Jason Reed and Benjamin C. Pierce. The intended application there was to differential privacy, where programs (interpreted as queries on a database), are constrained to have limited dependence on small changes to the input. Roughly speaking, limiting the precise dependence of the output on the exact values input ensures that the privacy of the people whose information is in the database is protected. The Reed-Pierce paper presents a linear type system that treats information leakage as a resource that needs to be conserved. The model I presented in the talk doesn’t mention linearity anywhere at all. However, it can be encoded in the reflexive graph model by treating the Reed-Pierce resource-limited types as types indexed by Dist, and building up the other connectives using the same techniques as the Day-construction models of Bunched Implications.
Geometric-group indexed types first appeared in Abstraction and Invariance for Algebraically-indexed Types by myself, Patricia Johann, and Andrew Kennedy. They are a generalisation of the unit-indexed types from Andrew’s previous work, but allow for a richer set of transformations that the program can be invariant under. We looked at two other forms of “algebraically-indexed types”: one for information flow, and one for distance indexing. The indefinability results from Andrew’s work carry over to this more general setting too.
The classical mechanics types, terms, and examples are taken from my paper From Parametricity to Conservation Laws, via Noether’s Theorem. Emmy Noether’s Theorem is a result in the Calculus of Variations that shows that it is possible to derive conserved properties of the stationary solutions of variational problems from continuous symmetries of their actions, and in some cases to go back again. This theorem is very important in physics. Many physical theories, not just in classical mechanics, are described in terms of stationary solutions of variational problems. Noether’s theorem gives a way to deduce properties of solutions without having to explicitly solve the system. In computer science terms, roughly speaking, Noether’s Theorem shows how to derive time-global invariants over runs of a system from time-local invariants of the single-step behaviour of the system (though in Noether’s case, there are no “single-steps” because everything is continuous). My paper showed that it was possible to derive the local symmetries as consequences of free theorems, which can then be plugged into Noether’s theorem. Note that I didn’t show that parametricity implies Noether’s theorem, though the idea doesn’t necessarily seem too outlandish, if one could give a suitable account of continuous change. Noether’s original paper Invariant Variation Problems is available in translation. I found the book Applications of Lie Groups to Differential Equations by Peter Olver extremely helpful in understanding the basic theorem, and its generalisations, which were also proved by Noether. The book Emmy Noether’s Wonderful Theorem gives a readable account of the background and how the theorem is applied in physics applications.
The non-example of Higher Order Abstract Syntax is from my paper Syntax for Free: Representing Syntax with Binding using Parametricity, which proved that the (denotation of the) type forall a. ((a → a) → a) → (a → a → a) → a is isomorphic to the set of closed lambda-terms. This result appears to require Kripke-indexed relations, which are not included in the simple reflexive graph model I used in the talk. After my talk, Ambrus Kaposi mentioned some ideas to me of how to incorporate Kripke-indexed relations, which I need to follow up.
The final section of the talk reported on unpublished work in progress on internalising relational parametricity into the type theory. Using the reflexive graph model at a meta-theoretical level is all very well, but it would be extremely useful to be able to prove invariance results internally, just as one can prove equalities between programs internally in dependent type theories. The approach I presented here is based on the observation that “relatedness” is sort of like a weakened version of equality in Martin-Löf type theory (MLTT). (For simplicity I only considered MLTT with equality reflection.) The resulting system is something like Ambrus Kaposi and Thorsten Altenkirch’s Cubical Type Theory, but differs in how it considers equality within in the universe U of small types. Jean-Phillipe Bernardy, Thierry Coquand, and Guilhem Moulin have published a syntax and model for internalised parametricity. Their system has unary, but higher-dimensional, relations, and does not (as far as I understand) have an explicit relationship with equality.

After the talk there were a few questions, which were written down on 5x3 index cards and read out by the session chair. This style of questions is an innovation by Lindsey for OBT to try to encourage people who wouldn’t normally ask questions to ask, and to share time more equally between questions. My impression was that possibly more questions were asked than I would have normally expected. Also, I got to keep the cards with the questions on:

The questions, and my answers, are listed below. These answers are with the benefit of hindsight and a long plane journey to ruminate on “what I should have said”, so they aren’t exactly what I said at the time.

How is that related to homotopy type theory/univalence? This is an especially natural question given the formal similarities between the higher-dimensional models for both. I used to think that relationally parametric type theory would be a “baby” version of univalent type theory, where the composable, reversible paths between objects are replaced by a weaker notion of relatedness. However, I now think that relational parametricity and homotopy type theory are orthogonal. Relational Parametricity is agnostic with respect to the notion of equality it uses, and so it is possible to speak about relationally parametric (extensional type theory | type theory with equality reflection | intensional type theory | observational type theory | univalent type theory). However, thinking a bit more, it might be possible that Homotopy Type Theory might be a sub-system of relationally parametric type theory, where relatedness just so happens to be slightly richer.
What ever came out of John Harrison’s “WLOG” work? Any useful tools for automatic theorem proving? I don’t know about tools for automatic theorem proving, though I think that it would be interesting (I vaguely remember that constraint solving systems often involve the use of symmetry to cut down the search space; the name Karen Petrie springs to mind). Harrison’s original work is now part of the HOL light theorem prover, in the Multivariate library.
Have you looked at Lorentz invariance? This is in reference to how all the examples I showed of mechanics systems were invariant under Galliean invariance, which doesn’t account for the effects of special relativity. Lorentz invariance does account for special relativity. I haven’t looked at Lorentz invariance explicitly, but I think that it can be handled by the system I presented.
Where can I get your slides and/or papers you used as examples? This page.
How about Calculus? At the time I interpreted this question as being about the special type constructor C^\infty I used to internalise smooth functions for the purposes of classical mechanics models, and I muttered something about Synthetic Differential Geometry. Looking back, I think it probable that the questioner was actually asking about the relationship between the “theory of change” model I talked about here, and differential calculus as a theory of continuous change. I think that this is a really interesting question, which might have a lot to say about incremental computation, bidirectional computation and reversible computation. The paper A theory of changes for higher-order languages — incrementalizing λ-calculi by static differentiation gives an account of a static “differentiation” process for lambda-calculus terms for the purposes of doing incremental computation. Gabriel Scherer noted a similarity between that work and relational parametricity, which I wrote a blog post about.

The Incremental λ-Calculus and Relational Parametricity

Thu, 23 Apr 2015 00:00:00 +0000

Back in February, the paper A theory of changes for higher-order languages — incrementalizing λ-calculi by static differentiation by Cai, Giarusso, Rendel, and Ostermann, was posted to Lambda-the-Ultimate. The poster, gasche, made the following parenthetical comment at the end of the L-t-U post:

(The program transformation seems related to the program-level parametricity transformation. Parametricity abstract over equality justifications, differentiation on small differences.)

In this post, I’m going to substantiate gasche’s conjecture by showing how (a very slight simplification of) the theory of changes presented in the paper is formally the same thing as relational parametricity. Where Reynolds’ relational parametricity for System F is a theory of how programs act under change of data representation — i.e., change of types; Cai *et al.*’s theory of changes is a theory of how programs act under change of values.

A Theory of Changes, and Change Structures

Cai et al. are interested in translating purely functional programs that map values of type A to values of type B into programs that map changes in values of type A to changes in values of type B. The intention is that the translated version of the program that maps changes to changes will more efficiently handle incremental updates to the input than doing a full recomputation.

In any attempt to define such a translation, you are going to need to define what you mean by “changes in the values of type A”. To this end, Cai et al. define the following notion of change structure. A change structure (for a particular type) consists of:

A set A of the possible values of this type;
For each value a ∈ A, a set ∂A(a) of possible changes that can be applied to the value a.
An operation ⊕ that takes a value a ∈ A and a change δa ∈ ∂A(a) and produces a value a ⊕ δa ∈ A that represents the result of applying the change δa to the value a.
An operation ⊖ that takes an a ∈ A and a b ∈ A and produces a change b ⊖ a ∈ ∂A(a) that represents the change required to get from a to b.

These operations are subject to several laws that are laid out in Section 2 of the paper.

Requiring the existence of the ⊖ operation for every change structure seems to be quite strong to me. By requiring ⊖, we are requiring that for any two values a and b, there must be at least one change that connects them, and for some reason we have chosen this change out of possibly many choices as “special”. Just by requiring the existence of a change that translates us from a to b, we are making a commitment that the “space” of A values is connected in some way, which may not be justified in some applications. What if there are states that we can get into that are not get-out-able? What if we only want to consider changes that are appending of new data?

As far as I can see, the ⊖ operation is only used in the paper to define the zero change 0 ∈ ∂A(a) for every value a ∈ A, and then is never mentioned again. To me, the existence of a zero change for every value seems much easier to justify than a general ⊖ operation. Therefore, I propose the following mild generalisation of change structure, which only differs from the Cai *et al.*’s definition in the final point:

A set A of possible values;
For each value a ∈ A, a set ∂A(a) of possible changes that can be applied to the value a.
An operation ⊕ that takes a value a ∈ A and a change δa ∈ ∂A(a) and produces a value a ⊕ δa ∈ A that represents the result of applying the change δa to the value a.
For every a ∈ A, a zero change 0 ∈ ∂A(a).

Change structures allow Cai et al. to define the notion of a derivative of a function f. The idea is that if f is the original function, then f' is a function that can compute changes in the output, given the original input value and the change in the input value.

Formally, given a pair of change structures, (A, ∂A, ⊕, 0) and (B, ∂B, ⊕, 0), and a function f : A → B Cai et al. define a derivative of f to be a function f' : (a : A) → ∂A(a) → ∂B(f a) such that: i) f(a ⊕ δa) = (f a) ⊕ (f' a δa); and ii) f' a 0 = 0. (Notes: point ii is actually a lemma in the paper, which I think is provable if you have ⊖, but I have to state explicitly; also, the paper calls f' the derivative, but in general it isn’t unique for a given f.)

Given these definitions, we can define a category of change structures: the objects are change structures, and the morphisms are pairs (f,f') of functions f between the sets of values, and derivatives f'. Looking at change structures as a category, we can see that the middle four pages of Cai *et al.*’s paper, from Section 2.2 to the end of Section 3, essentially boils down to proving that this category is cartesian closed, and so can model the simply typed λ-calculus.

Relational Parametricity via Reflexive Graphs

Relational parametricity is usually introduced in terms of logical relations. A logical relation is a relation between two interpretations of the same syntactic type A that is defined by looking at the “logical” structure of the type A (that is: looking at the “logical” constructors →, +, × and so on used to construct A). The general idea is that we relate two interpretations of the same type A to capture the idea of “change of representation” for that type.

An alternative approach to relational parametricity, that ends up being equivalent when we are just talking about types uses reflexive graphs. This approach is originally due to Robinson and Rosolini in their “Reflexive Graphs and Parametric Polymorphism” paper from 1994. The idea is to abstract logical relations away from types and relations, and to look at just the bare structure of what we are defining. A reflexive graph consists of:

A set O, which we will call abstract objects.
A set R, which we will call abstract relations or edges.
A function src : R → O (“source of an edge”) that maps each edge to its source object.
A function tgt : R → O (“target of an edge”) that maps each edge to its target object.
A function refl : O → R (“reflexive edge”) that maps each object o to a reflexive edge with source o and target o.

Conceptually, we can think of the objects in O as the actual data we compute with, and the edges in R as telling us about abstract relationships between pairs of data items. A reflexive graph morphism will tell us how to map data to data and relationships to relationships, in such a way that the two mappings agree.

Formally, a morphism between two reflexive graphs (O₁, R₁, src₁, tgt₁, refl₁) and (O₂, R₂, src₂, tgt₂, refl₂) consists of a pair of functions f : O₁ → O₂, mapping objects to objects, and r : R₁ → R₂, mapping relations to relations, such that the three operations src, tgt and refl are preserved.

As with change structures, we can make reflexive graphs and reflexive graph morphisms into a category, which is cartesian closed for general reasons (specifically, the category of reflexive graphs is a presheaf category, which are always cartesian closed).

I think it is useful to consider reflexive graphs because they are a larger world where normal logical relations live, alongside many other interesting things. We can define a particular reflexive graph that captures the idea of “relation between two interpretations of a type”, but we can also go on to define other reflexive graphs to represent other things, like higher kinds, or geometric transformation groups, and, as I’ll demonstrate below, Cai *et al.*’s change structures.

In more detail, the reflexive graph that captures the idea of “relation between two interpretations of a type” is defined as having objects that are (small) sets, edges that are triples (A,B,R), where A and B are small sets, and R is binary relation between A and B. We define src(A,B,R) = A, tgt(A,B,R) = B. The “reflexive edge” for each small set is defined to be the equality relation: refl(A) = (A,A,{(a,a) | a ∈ A}). Making the reflexive edge from every set to itself be equality captures the Identity Extension property identified by Reynolds.

For interpreting System F, and the higher kinded variant System Fω, we interpret kinds (i.e., *, * → *, (* → *) → * and so on) as reflexive graphs, types (possibly with free variables) as morphisms of reflexive graphs, and terms as transformations between morphisms of reflexive graphs. See my paper Relational Parametricity for Higher Kinds for more details.

When interpreting dependent types in a relationally parametric model, we get to make a simplification: we unify the interpretations of types and kinds. Now all types are interpreted as (families of) reflexive graphs, and terms are interpreted as (families of) reflexive graph morphisms. The type of small types, U, is interpreted using the reflexive graph Type I gave the definition of above. See the paper A Relationally Parametric Model of Dependent Type Theory, that I wrote with Neil and Patty, for more details.

So, the category of reflexive graphs is the natural place for interpreting relational parametricity for type systems that are more expressive than plain System F.

Change Structures and Reflexive Graphs are equivalent

I’m now going to show that Cai *et al.*’s change structures (or at least my slight variant) and reflexive graphs are the same thing. More precisely, the categories involved are equivalent, but here I’ll just stick to showing how to convert a change structure to reflexive graph and back.

Given a change structure (A, ∂A, ⊕, 0), we define a reflexive graph as follows:

The set of objects is the set A.
The set of relations is the set consisting of pairs (a, δa), where a ∈ A and δa ∈ ∂A(a).
The source map is defined as src(a, δa) = a.
The target map is defined as tgt(a, δa) = a ⊕ δa.
The reflexive maps is defined as refl(a) = (a,0).

Conversely, given a reflexive graph (O, R, src, tgt, refl), we define a change structure as follows:

The set of values is O.
For each value o ∈ O, the set of changes at o is ∂O(o) = { r ∈ R | src(r) = o } — i.e., the set of relations with source o.
The ⊕ operation is defined as o ⊕ r = tgt(r) — i.e., we take the target of a relation whose source is o.
For each o ∈ O, the 0 change in ∂O(o) is refl(o).

If we start with either a change structure or a reflexive graph then translating to the other and back again will always give us something that is isomorphic to what we started with. After a bit more work, we can see that this will give us an equivalence of categories.

So what?

So we’ve got an equivalence of categories: one intended for interpreting incremental recomputation, and one intended for interpreting relational parametricity. What does this give us?

Since the category of reflexive graphs can interpret dependent type theory (see here), the fact that the categories of change structures and reflexive graphs are equivalent means that we automatically have a notion of static differentiation of programs in the sense of Cai et al. for dependently typed languages.
I think this equivalence gives us a way to think about relational parametricity. Relational parametricity (and “theorems for free”) are sometimes characterised as holding because programs “cannot inspect their type parameters”, or “don’t know what type is being used”. But the equivalence with change structures indicates to me that thinking of parametricity as a theory of how programs act under changes of their parameters. This viewpoint is explicit in Kennedy’s Relational Parametricity and Units of Measure from 1997, where he treats polymorphism over dimensions as invariance under scaling. However, the “theory of change” viewpoint does not seem to be a common way to think about parametricity or polymorphism in the literature in general.
I hope that a unified viewpoint on parametricity and change structures may lead to interesting advances in both. As one example: what about having additional structure on the set of changes? Could we arrange changes into a partial order to say that one change was “bigger” than another? This might give us a way of stating that an incremental recomputation gives us the least change in the output for a given change in the input. Then what does that mean for parametricity? Another example: if the derivative function maps changes in the input to changes in the output, then can we formulate what is required of a function that maps changes back? (Fun exercise: show that “well-behaved lenses” are equivalent to change structure morphisms between certain change structures, where the derivative function partially applied to a value — f' a : ∂A(a) → ∂B(f a) — always has an inverse.)

Slides for “An Algebraic Approach to Typechecking and Elaboration”

Sun, 19 Apr 2015 00:00:00 +0000

Two months at the Scottish Programming Languages Seminar, February 2015 Strathclyde Edition, I gave a talk entitled “An Algebraic Approach to Typechecking and Elaboration”. Several people have asked me to put the slides online, so here they are:

The point of the talk was to present an algebraic approach to specifying type systems ─ algebraic in the sense of algebraic theories with operations and equations. I reckon this presentation leads naturally to a unification of the way that typed functional languages like ML and Haskell elaborate their source language into explicitly typed core languages, and the tactic languages in interactive theorem provers like Coq and Isabelle.

Update: (2015-05-20) I gave this talk again at the Workshop on Type Inference and Automated Proving in Dundee, and it was recorded (link to video).

Update: (2015-09-03) I gave this talk a third time at the Higher Order Programming with Effects (HOPE) workshop colocated with ICFP 2015 in Vancouver, and it was again recorded (link to video).

Propositions as Filenames, Builds as Proofs: The Essence of Make

Fri, 17 Apr 2015 00:00:00 +0000

The make program is a widely used tool for building files from existing files, according to a set of build rules specified by the user. It is usually used to compile executable programs from source code, but can also be used for many other jobs where a bunch of things are generated from other things, like this website, for example.

Many alternatives to make have been proposed. Motivations for replacing make range from a desire to replace make's very Unix-philosophy inspired domain-specific language for describing build rules (Unix-philosophy in the sense that it often works by coincidence, but falls over if you do something exotic, like have filenames with spaces in, or have an environment variable with the “wrong” name), or make's slowness at some tasks, or a perception that make doesn't treat the make-alternative implementor's favourite programming language with the special treatment it so obviously deserves.

Nevertheless, I think that make (or at least the GNU variant I am most familiar with) has a core essence that can be profitably extracted and analysed.

The Essence of `make`

The essence of make is this: make is an implementation of constructive logic programming, using the following instantiation of the “Propositions-as-X” paradigm:

Atomic propositions are filenames. The filenames main.c, main.o and myprogram are all examples of atomic propositions in make's logic. For make, the idea of “well-formed formula” from traditional logic means “doesn't have spaces in”.
Compound propositions are build rules. A build rule that states that myprogram can be built from main.o and module.o is a statement that the atomic propositions main.o and module.o imply myprogram. Pattern rules like %.o: %.c; gcc -o $@ -c $< are universally quantified compound propositions: this says that, for all x, the atomic proposition x.c implies the atomic proposition x.o. Static pattern rules are essentially a form of bounded quantification.
Note that the form of compound propositions allowed is extremely restricted, even by the standards of logic programming: we are allowed at most one universal quantifier, which quantifies over space-less strings, the rest of proposition must be of the form “f1 and f2 and ... and fn implies g”, and if there is a quantifier, the variable must appear in the goal formula g. This format corresponds to a restricted form of Horn Clauses, as used in normal logic programming.

If we stopped here, then make would not be any more than an extremely restricted form of Prolog. But what makes make special is that it implements a constructive logic: it generates proofs, or evidence, for the propositions it proves.

Proof, or evidence, of an atomic proposition somefile is the content of an actual file somefile in the filesystem. Some evidence is provided by the user, in the form of source files. Evidence for deduced atomic propositions, e.g., .o files, is generated by the proofs for compound propositions:
Proof of a compound proposition “x and y implies z” is a command to run that will generate the proof of the atomic proposition z from the proofs of the atomic propositions x and y. For pattern rules, this proof is parameterised by the instantiation of the universally quantified variable. For some reason, in make, the universally quantified variable is written as “%” in the proposition, and “$*” in the proof.

What `make` does

Using this mapping between logic and make, I see make as conceptually performing three tasks when it is told to use Makefile to generate the target myprogram.

It executes the Makefile, expanding out variables. This generates a collection of build rules.
It constructs a proof of the atomic proposition myprogram using backward-chaining proof search from the goal, via the build rules (aka compound propositions), back to the evidence for atomic propositions provided by the user in the file system. In traditional logic, this proof would be represented using a tree, but obvious efficiency gains can be had by exploiting sharing and representing it as a directed acyclic graph.
It executes the proof to generate the evidence of the atomic proposition myprogram. The evidence for the provability of myprogram is a file myprogram in the filesystem, generated by the proofs of the build rules and source files it's proof depends on. This step can often be made more efficient by reusing existing pieces of evidence if the evidence they were built from hasn't changed.

The GNU make manual's description combines the last two steps into one “run-the-build” step, and in practice this is what an realistic implementation ought to do. (And the first step is, in GNU make's reality, more complex because make can rebuild included files and restart itself, but I'm glossing over that for now.)

So what?

I think that there are real benefits to seeing make-like systems as implementations of constructive logic programming:

I believe that seeing make-like systems as a form of constructive logic programming elucidates the differences between some of the make alternatives that have been proposed. For instance, I think that the Ninja system essentially gets its speed ups by caching the some of results of the proof search step by storing the expansions of all of the universally quantified build rules that are needed. The OMake system allows for targets to dynamically depend on dependencies listed in generated files, via “scanner” dependencies. I think this corresponds to proof search in a dependently-typed logic that allows propositions to depend on the generated evidence of other propositions.
We can start to look at makes's restrictions through the lens of logic programming, and start to think about more expressive build tools:
- Why are build rules restricted to at most one universal quantifier? What would we gain my allowing unrestricted Horn clauses? What if the universally quantified variables didn't have to appear in the goal formula, as in most other logic programming languages?
- Build rules that generate multiple files are Horn clauses with conclusions that are conjunctions (ands) of atomic propositions. GNU make and others make multiple targets difficult, but from a logical point-of-view, there is no problem.
- What rules does make use to resolve the choice between multiple proofs of the one atomic proposition? Could we have build systems that produce sets of proofs for each proposition? Can this be used to do multi-platform builds? Can we assign weightings to build rules so that make picks the overall “best” proof/build strategy?
- make implements a “top-down” approach to evaluating its logic program. Why not also implement a “bottom-up” evaluation too? This would enable us to ask questions like “what can be built using these rules and these source files?”. This might finally enable decent TAB-completion for make at the command line, and IDE introspection capabilities.
- Can logics that incorporate forms of (sound) circular reasoning be used to do build jobs that require iteration until a fixpoint. Can we use fixpoint logic to work around the tragedy of LaTeX's “Label(s) may have changed. Rerun to get cross-references right.”?
- Do all atomic propositions have to be filenames? Why not URLs? Why not proof-irrelevant ephemeral propositions?
- Can the connection between logic programming and relational databases be used? Can we use a make-like tool to query a database, and generate reports?
- Can we automatically augment the proof graphs that make implicitly generates to add provenance information?

Interesting stuff, I think.

POPL Slides

Wed, 29 Jan 2014 00:00:00 +0000

I gave two talks at POPL 2014, back to back. This was pretty frightening beforehand, but seemed to go alright.

Here are the slides:

From Parametricity to Conservation Laws, via Noether's Theorem

A Relationally Parametric Model of Dependent Type Theory

One Done, Two Submitted

Wed, 17 Jul 2013 00:00:00 +0000

One paper finished, two new ones submitted.

Productive Coprogramming

Conor and I have just submitted the final version of "Productive Coprogramming with Guarded Recursion" to the publishers. Looking forward to ICFP in Boston!

Pair of Papers Pertaining to Parametricity

Dependent Types

With Neil and Patty, we've constructed a relationally parametric model of impredicative and predicative dependent type theories. Highlights: (1) we prove the existence of initial algebras for all indexed functors; and *(2) the model is constructed using reflexive graphs, a kind of cut-down version of groupoids.

Classical Mechanics

Just by myself, I've also done a paper on using relational parametricity to prove symmetry properties of Lagrangians in classical mechanics, so that you can derive conservation laws via Noether's theorem. The main point is to show that parametricity isn't just for computer programs; physics looks like it is ripe with opportunities for applications of theorems for free.

Productive Coprogramming with Guarded Recursion

Fri, 29 Mar 2013 00:00:00 +0000

Conor McBride and I have just submitted a new paper to ICFP. In it, we attempt to use Nakano-style guarded recursion to write productive coprograms.

This is an elaboration of Conor's blog post and the slides I posted here a while ago.

Here's a link to the paper, and the abstract:

Total functional programming offers the beguiling vision that, just by virtue of the compiler accepting a program, we are guaranteed that it will always terminate. In the case of programs that are not intended to terminate, e.g., servers, we are guaranteed that programs will always be productive. Productivity means that, even if a program generates an infinite amount of data, each piece will generated in finite time. The theoretical underpinning for productive programming with infinite output is provided by the category theoretic notion of final coalgebras. Hence, we speak of coprogramming with non-well-founded codata, as a dual to programming with well-founded data like finite lists and trees.

Systems that offer facilities for productive coprogramming, such as the proof assistants Coq and Agda, currently do so through syntactic guardedness checkers. Syntactic guardedness checkers ensure that all self-recursive calls are guarded by a use of a constructor. Such a check ensures productivity. Unfortunately, these syntactic checks are not compositional, and severely complicate coprogramming.

Guarded recursion, originally due to Nakano, is tantalising as a basis for a flexible and compositional type-based approach to coprogramming. However, as we show, by itself, guarded recursion is not suitable for coprogramming due to the fact that there is no way make finite observations on pieces of infinite data. In this paper, we introduce the concept of clock variables that index Nakano’s guarded recursion. Clock variables allow us to ``close over’’ the generation of infinite data, and to make finite observations, something that is not possible with guarded recursion alone.

Abstraction and Invariance for Algebraically Indexed Types

Wed, 07 Nov 2012 00:00:00 +0000

Patricia Johann, Andrew Kennedy and I have a new paper that will be presented at POPL in January! This paper is an extension of Andrew's POPL'97 paper on interpreting dimension types in terms of scaling invariance. Here's a link to the paper and the abstract:

Reynolds' relational parametricity provides a powerful way to reason about programs in terms of invariance under changes of data representation. A dazzling array of applications of Reynolds' theory exists, exploiting invariance to yield “free theorems”, non-inhabitation results, and encodings of algebraic datatypes. Outside computer science, invariance is a common theme running through many areas of mathematics and physics. For example, the area of a triangle is unaltered by rotation or flipping. If we scale a triangle, then we scale its area, maintaining an invariant relationship between the two. The transformations under which properties are invariant are often organised into groups, with the algebraic structure reflecting the composability and invertibility of transformations.
In this paper, we investigate programming languages whose types are indexed by algebraic structures such as groups of geometric transformations. Other examples include types indexed by principals--for information flow security--and types indexed by distances--for analysis of analytic uniform continuity properties. Following Reynolds, we prove a general Abstraction Theorem that covers all these instances. Consequences of our Abstraction Theorem include free theorems expressing invariance properties of programs, type isomorphisms based on invariance properties, and non-definability results indicating when certain algebraically indexed types are uninhabited or only inhabited by trivial programs. We have fully formalised our framework and most examples in Coq.

Theorems for Free

Wed, 07 Nov 2012 00:00:00 +0000

I gave a talk last night at the Ed Lambda, the Edinburgh functional programming meetup, on “Theorems for Free”. This was (I hope) a fairly high-level talk about how free theorems are derived, and some extensions to other kinds of polymorphism that I've worked on recently. Here are the slides I used:

I didn't include much in the way of references to the literature in the talk, but the main paper to look at if you are interested is Phil Wadler's classic Theorems for Free! (warning: PostScript!).

Thanks for Rob Stewart for organising things!

Interleaving Data and Effects

Thu, 06 Sep 2012 00:00:00 +0000

Patricia Johann, Neil Ghani, Bart Jacobs and myself have just submitted a paper on interleaving pure data types with effects. This is a much more detailed version of the blog post I wrote back in January on reasoning about stream processing with effects.

Here is the abstract:

The study of programming with and reasoning about inductive datatypes such as lists and trees has benefited from the simple categorical principle of initial algebras. In initial algebra semantics, each inductive datatype is represented by an initial f-algebra for an appropriate functor f. The initial algebra principle then supports the straightforward derivation of definitional principles and proof principles for these datatypes. This technique has been expanded to a whole methodology of structured functional programming, often called origami programming.
In this article, we show how to extend initial algebra semantics from pure inductive datatypes to inductive datatypes interleaved with computational effects. Inductive datatypes interleaved with effects arise naturally in many computational settings. For example, incrementally reading characters from a file generates a list of characters interleaved with input/output actions. Straightforward application of initial algebra techniques to effectful datatypes leads to unnecessarily complicated reasoning, because the pure and effectful concerns must be considered simultaneously. We show how these concerns can be separated the abstraction of initial f-and-m-algebras, where the functor f describes the pure part of a datatype, and the monad m describes the interleaved effects. Because initial f-and-m-algebras are the analogue for the effectful setting of initial f-algebras, they support the extension of the standard definitional and proof principles to the effectful setting. Because initial f-and-m-algebras separate pure and effectful concerns, they support the direct transfer of definitions and proofs from the pure setting to the effectful setting.
Initial f-and-m-algebras are originally due to Filinski and Støvring, and were subsequently generalised to arbitrary categories by the authors of this article. In this article, we aim to introduce the concept of initial f-and-m-algebras to a general functional programming audience.

The actual paper is available here.

Relational Parametricity for Higher Kinds

Wed, 05 Sep 2012 00:00:00 +0000

I just gave a talk at CSL 2012 on "Relational Parametricity for Higher Kinds". In the paper I explain how to extend the usual relationally parametric models of polymorphic types to handle higher kinded types, like the ones found in Haskell and Scala. As a consequence, you get encodings of things like equality types, higher-kinded existential types and higher-kinded initial algebras, with nice reasoning principles.

The slides:

The paper, with abstract, is available here.

Reasoning about Stream Processing with Effects

Fri, 06 Jan 2012 00:00:00 +0000

It is a truth, universally acknowledged, that any programming technique must be in want of a reasoning principle.

Stream processing in Haskell is very much in the air at the moment, what with Iteratees (as embodied in the Enumerator library), Conduits and probably some more that I don't know about.

Patty Johann, Neil Ghani, Bart Jacobs and I have recently had the paper "Fibrational Induction Meets Effects" accepted to FoSSaCS 2012 that turns out to be very relevant to discussing and reasoning about the kinds of stream processing problems that these Haskell libraries seek to resolve.

In the paper we build upon the work of Andrzej Filinski and Kristian Støvring on induction principles for data structures that interleave pure data with effects, where the effects are defined in terms of some monad. Filinski and Støvring gave a induction principle for such interleaved effectful data types in a small programming language with effects and fixed notion of data type and predicate. In our FoSSaCS paper, we have generalised this to a more general categorical setting, and also generalised the notion of predicate that can be considered (so you can have proof-relevant predicates, or Kripke predicates, for instance).

Our paper is quite technical, since we need the structure of fibrations and so on to properly attain the right level of generality. Nevertheless, the core principle is a relatively simple and revolves around a recursion scheme for pure data interleaved with monadic effects (which generalises the one given by Filinski and Støvring). This turns out to have direct application to reasoning about the processing of streaming data.

Effectful Streams

Effectful streams generate potentially infinite amounts of data by executing effects as they do so. For example, an effectful stream may produce data by reading it from a network socket. An effectful stream is nothing more than an interleaving of some monad "m" with news of whether the stream has stopped or has yielded an element. This can be expressed by the following Haskell declarations of a pair of mutually recursive types:

newtype Stream m a = Stream { forceStream :: m (StreamStep m a) }

data StreamStep m a
    = StreamEmit a (Stream m a)
    | StreamStop

Constructing streams with no (new) effects is accomplished by just using the return of our chosen monad, and the appropriate constructor:

nil :: Monad m => Stream m a
nil = Stream $ return StreamStop

cons :: Monad m => a -> Stream m a -> Stream m a
cons a s = Stream $ return (StreamEmit a s)

Given a value stream of type Stream m a we can kick it with forceStream to give us a monadic action that will yield either StreamEmit a moreStream or StreamStop. The stream gets to execute some effect in the monad m before telling us whether the stream has ended or not. It can use this monadic effect to do things like read from files, consult random number sources or use it as a side channel to report errors.

The stream append function shows how a stream can be interrogated by a recursive function:

append :: Monad m => Stream m a -> Stream m a -> Stream m a
append s1 s2 = Stream $ do
  streamStep <- forceStream s1
  case streamStep of
    StreamEmit a s1' -> forceStream (cons a (append s1' s2))
    StreamStop       -> forceStream s2

Streams look a lot like normal Haskell lists, except that rather than the ambient Haskell effect of "possibly a non-terminating black hole", we have the effects described by the monad m (plus the non-optional "possibly a non-terminating black hole" effect). For instance, we can define a stream that emits characters read from a Handle until it hits the end of file:

ofHandle :: Handle -> Stream IO Char
ofHandle handle = loop
  where
    loop = Stream $ do
      isEOF <- hIsEOF handle
      if isEOF then do hClose handle
                       return StreamStop
               else do c <- hGetChar handle
                       return (StreamEmit c loop)

Stream Readers

Streams constitute the supply-side of data processing. We can use a similar pattern to treat the consumption side. Readers are defined as follows, and look very similar to Iteratees:

newtype Reader a m b = Reader { forceReader :: m (ReaderStep a m b) }

data ReaderStep a m b
    = ReaderRead (Maybe a -> Reader a m b)
    | ReaderEmit b

Similar to Streams, a value of type Reader a m b can be prodded with forceReader. It will then tell us (after some monadic effect) whether it wants to read some data (ReaderRead) or if it has finished reading and wants to output some data (ReaderEmit). The Maybe type constructor in ReaderRead is intended to indicate whether or not end-of-stream has been reached.

The obvious thing to do now is to connect up a Stream and a Reader to feed the data from one into the other. There are actually (at least) two different ways of doing this, depending on whether we execute the effects of the stream before the reader, or the other way round. Executing the stream before the reader gives the following definition:

(|>|) :: Monad m => Stream m a -> Reader a m b -> m b
s |>| r = do
    streamStep <- forceStream s
    case streamStep of
      StreamEmit a s' -> do
        readerStep <- forceReader r
        case readerStep of
          ReaderRead k -> s' |>| k (Just a)
          ReaderEmit b -> return b
      StreamStop -> do
        runOnNothing r

If the stream stops early by returning StreamStop, then we use the following function runOnNothing to feed Nothing to a Reader until it yields a value:

runOnNothing :: Monad m => Reader a m b -> m b
runOnNothing r = do
    readerStep <- forceReader r
    case readerStep of
      ReaderRead k -> runOnNothing (k Nothing)
      ReaderEmit b -> return b

Generic Interleaved Data and A Recursion Scheme

I will now show that the Stream and Reader types may be generalised to a common pattern. By exploiting this common pattern, we obtain a powerful recursion principle for data interleaved with effects. This recursion principle also comes equipped with a reasoning principle, which allows us to prove things about functions that recurse over data interleaved with effects.

The common shape of the Stream and Reader types is captured by the following pair of Haskell type declarations. All the parts specific to Streams or Readers have been abstracted out into the argument f :: * -> * (which we will assume is an instance of the Functor type class).

data Step m f = Step (f (D m f))

newtype D m f = D { force :: m (Step m f) }

A value of type D m f is therefore an interleaving of the effects described by m with the pure data described by f. The function force plays the part of the forceStream and forceReader functions defined above.

Effectful data can be constructed and deconstructed by the following functions:

construct :: Monad m => f (D m f) -> D m f
construct x = D $ return (Step x)

deconstruct :: Monad m => D m f -> m (f (D m f))
deconstruct d = do Step x <- force d
                   return x

When using deconstruct there may be some effects to execute before we get access to the underlying pure data described by f.

To recover the Stream type, we simply define the appropriate f to describe the two things that Streams are allowed to do: emit values and cease to be.

data StreamStep a x
    = StreamEmit a x
    | StreamStop
    deriving Functor

type Stream m a = D m (StreamStep a)

In the new StreamStep type, the type parameter x indicates the hole where the next step of the recursion is placed.

The nil and cons constructors can then be defined in terms of construct and the appropriate part of the StreamStep type:

nil :: Monad m => Stream m a
nil = construct StreamStop

cons :: Monad m => a -> Stream m a -> Stream m a
cons a s = construct (StreamEmit a s)

The benefit of re-expressing the Stream type in this way is that we can now define a powerful recursion scheme on values of type D m f, including Stream m a, and use this to define functions like append and (|>|). The interesting part is that this recursion scheme comes equipped with a reasoning principle, which allows us to prove things about our functions. To properly define the recursion scheme we first need to know what an Eilenberg-Moore algebra for a monad is.

Eilenberg-Moore Algebras

For any monad m, an m-Eilenberg-Moore algebra consists of a pair of a type a and a function h :: m a -> a, satisfying some laws that state that h interacts nicely with the structure of the monad. I think of the existence of an Eilenberg-Moore algebra structure on a as roughly stating that the type a is effectful in the sense that it can have additional effects "prepended" on to it.

The existence of an Eilenberg-Moore algebra structure for a type can be expressed as a Haskell type class, EMAlgebra:

class (Functor m, Monad m) => EMAlgebra m a where
    algebra :: m a -> a

Of course we cannot state the laws, so we shall just commit to promising that they hold. The type class mechanism also ties us to giving at most one Eilenberg-Moore structure for each pair of a monad m and a type a, where in fact there may be many, but this is a small price to pay for the convenience that type classes provide.

Every type of the form m a for some monad m is obviously effectful, so we can give it an Eilenberg-Moore algebra structure using the function join :: m (m a) -> m a defined in the Control.Monad module:

instance (Functor m, Monad m) => EMAlgebra m (m a) where
    algebra = join

This is the free Eilenberg-Moore algebra for the monad m and the type a.

The property of having an Eilenberg-Moore algebra is also preserved by the construction of function types. If b has an Eilenberg-Moore structure with respect to m, then so does a -> b for any a:

2012-01-07 13:15 Unfortunately, this instance and the previous one overlap, so GHC needs -XIncoherentInstances to handle it. This seems to work for the examples here, but probably a better solution is needed in general.

instance (EMAlgebra m b) => EMAlgebra m (a -> b) where
    algebra x a = algebra $ do f <- x; return (f a)

Finally, every one of our interleaved data and effects types carries a default Eilenberg-Moore structure, achieved by prepending the new effect before the first effect of the data:

instance (Functor m, Monad m) => EMAlgebra m (D m f) where
    algebra x = D $ x >>= force

Note that we now have two algebra structures on D m f: the m-Eilenberg-Moore structure defined here, and the f-algebra structure defined by the construct function above.

2012-01-07 13:15 Fixed a typo here, thanks to ehird in the comments below for spotting it.

A Recursion Scheme

A basic recursion scheme is the catamorphism, which allows us to eliminate an element of a recursive type Rec f using an algebra f a -> a, where Rec f is defined as follows:

data Rec f = In (f (Rec f))

The catamorphism function, or to use its common name fold, has the following type:

fold :: Functor f => (f a -> a) -> Rec f -> a

A function of type f a -> a is the (structure part) of an f-algebra. f-algebras are similar to Eilenberg-Moore algebras, except that they do not need to satisfy any laws (because we do not assume that f is a monad).

The fold function itself satisfies the following law (which can also be taken to be the definition):

fold h (In x) = h (fmap (fold h) x)

and is the unique function of type Rec f -> a to do so. This uniqueness property can be used to reason about functions built from fold.

To define functions that eliminate our interleaved effects and data types D m f, we could just use the fact that they are equivalent to the form m (Rec (f :.: m)), where - :.: - indicates composition of functors, and use fold on the type Rec (f :.: m). However, in doing this we are forced to consider the pure and effectful parts of our data simultaneously.

A better approach, which is derivable from the basic fold operator, is to eliminate values of type D m f using f-and-m-algebras. That is, we assume that the result type a has a an f-algebra structure f a -> a and an Eilenberg-Moore structure m a -> a. In this way, we can separate the pure and effectful parts of our recursion. By making use of the EMAlgebra type class defined above, we do not need to explicitly mention the effectful part at all.

Our new effectfulFold combinator has the following type, and I have given the direct Haskell implementation. It is an interesting exercise to see now it can be defined in terms of the fold combinator on Rec (f :.: m).

2012-01-07 13:15 Thanks to spacespacecomma on reddit for pointing out that the original definition here didn't type check. I was missing the applications of force.

effectfulFold :: (Functor f, EMAlgebra m a) =>
                 (f a -> a)
              -> D m f
              -> a
effectfulFold h = algebra . fmap loop . force
  where
    loop (Step x) = 
      h $ fmap algebra $ fmap (fmap loop) $ fmap force $ x

As we prove in the paper, generalising Filinski and Støvring's proof, functions defined using effectfulFold satisfy the following two properties. First, they preserve f-algebras, taking the f-algebra construct to the supplied f-algebra h:

effectfulFold h (construct x) = h (fmap (effectfulFold h) x)

Moreover, they preserve m-Eilenberg-Moore algebras, taking the default Eilenberg-Moore structure on D m f to the implicitly provided Eilenberg-Moore structure on a:

effectfulFold h (algebra x) = algebra (fmap (effectfulFold h) x)

Analogously to fold h, effectfulFold h is the unique function satisfying these two properties. In the paper, we use uniqueness to derive an induction principle for data interleaved with monadic effects, but here we can use uniqueness directly to reason about functions defined using effectfulFold.

Using the Recursion Scheme

The append function on streams that I defined by hand above can be re-expressed using the effectfulFold combinator:

append :: (Functor m, Monad m) => Stream m a -> Stream m a -> Stream m a
append s1 s2 = effectfulFold h s1
  where h (StreamEmit a s) = cons a s
        h StreamStop       = s2

By its definition in terms of effectfulFold we know that append preserves the Eilenberg-Moore structure on its first argument. This means that the following equation holds for x :: m (Stream m a)

append (algebra x) s2 = algebra (fmap (\s -> append s s2) x)

Note that append does not preserve the Eilenberg-Moore structure on its second argument. The Eilenberg-Moore structure for streams prepends effects on to the stream, but these effects will not be executed by append until after all the effects in the first stream have been executed.

Also by its definition in terms of effectfulFold, we know that append satisfies the following equations with respect to the "pure" part of streams:

append (construct StreamStop)        s2 = s2
append (construct (StreamEmit a s1)) s2 = construct (StreamEmit a (append s1 s2))

We can now use the uniqueness of functions defined by effectfulFold to see that append is associative. To show this, I'll use the uniqueness of functions defined by effectfulFold. We have two functions of type Stream m a -> Stream m a that we wish to prove equivalent:

\s1 -> append s1 (append s2 s3)

and

\s1 -> append (append s1 s2) s3

where the first one is defined directly in terms of effectfulFold. If we can show that the second function obeys the same properties as the first, i.e. that it preserves the m-Eilenberg-Moore algebra and the f-algebra used in the definition of append, then by uniqueness we will have shown that they are the same function.

It is easy to check that the second function preserves the Eilenberg-Moore structure on Stream m a, since it is just the composition of two functions that do so. So the meat of the proof is in showing that it preserves the f-algebra structure. This splits into two cases, one for StreamStop and one for StreamEmit. For the former, we must show that

append (append (construct StreamStop) s2) s3 = append s2 s3

but this follows directly from the properties we already know about append. The second case is a little harder. We must show that

  append (append (construct (StreamEmit a s1)) s2) s3
=
  construct (StreamEmit a (append (append s1 s2) s3))

This follows by repeated application of our knowledge about how append operates on input of the form construct (StreamEmit a s1).

Thus we have used the uniqueness property of functions defined using effectfulFold to show that append is associative.

Back to Readers

Just as the Stream type can be defined in terms of the generic D m f type, the Reader type can be defined in the same way:

data ReaderStep a b x
    = ReaderRead (Maybe a -> x)
    | ReaderEmit b
    deriving Functor

type Reader a m b = D m (ReaderStep a b)

The runOnNothing function can be defined in terms of effectfulFold, by recursing on the Reader argument:

runOnNothing :: (Functor m, Monad m) => Reader a m b -> m b
runOnNothing = effectfulFold f
  where f (ReaderRead k) = k Nothing
        f (ReaderEmit b) = return b

Likewise, the (|>|) function that connects a stream with a reader can be defined in terms of effectfulFold, by recursion on the stream argument, and using the Eilenberg-Moore structure on the type Reader a m b -> m b:

(|>|) :: (Monad m, Functor m) => Stream m a -> Reader a m b -> m b
(|>|) = effectfulFold f
  where f (StreamEmit a h) r = do
          readerStep <- deconstruct r
          case readerStep of
            ReaderRead k -> h (k (Just a))
            ReaderEmit b -> return b
        f StreamStop r = runOnNothing r

The (|>|) function executes the effects of the stream before the effects of the reader, letting the stream dictate how events proceed. An alternative is to execute the effects of the reader first, by defining the function in terms of recursion on the reader argument, again using effectfulFold:

(|>>|) :: (Monad m, Functor m) => Stream m a -> Reader a m b -> m b
s |>>| r = effectfulFold f r s
  where f (ReaderRead k) s = do
            streamStep <- deconstruct s
            case streamStep of
              StreamEmit a s' -> k (Just a) s'
              StreamStop      -> k Nothing nil
        f (ReaderEmit b) s = return b

There are obviously many other possibilities for connecting Streams to Readers. If the Stream finishes early, we could return the uncompleted Reader rather than passing it on to runOnNothing. Likewise, if the Reader emits a value before all the Stream has been read, we could return the leftover stream instead of just dropping it on the floor. In any case, all these possibilities are definable in terms of effectfulFold.

Readers are an instance of the Monad type class as the following Haskell declaration witnesses:

2012-01-07 13:15 Thanks again to spacespacecomma on reddit for pointing out that this doesn't directly work. You need to wrap the Reader type in a newtype to get it to work. I'll leave it as-is to give the general idea.

instance Monad m => Monad (Reader a m) where
    return b = construct (ReaderEmit b)
    c >>= k  = effectfulFold f c
        where f (ReaderRead k) = construct (ReaderRead k)
              f (ReaderEmit b) = k b

Using the uniqueness property of effectfulFold it is possible to prove that this instance really does satisfy the monads laws. In fact, it is possible to show that Reader a m is the sum of the monad m with the free monad on the functor f x = Maybe a -> x. This is an instance of a fact originally observed by Hyland, Plotkin and Power (Theorem 4).

Processors

If Streams produce data and Readers consume data, then what goes in between? Processors! A Processor is an instance of the same interleaved data and effects pattern as Streams and Readers:

data ProcessorStep a b x
    = ProcessorRead (Maybe a -> x)
    | ProcessorEmit b x
    | ProcessorStop

type Processor a m b = D m (ProcessorStep a b)

A Processor, once kicked using force, can either ask for more input (ProcessorRead), can emit some output (ProcessorEmit) with a promise to carry on, or can signal end of stream (ProcessorStop). Processors are intended to fulfil the same role as Enumeratees or Conduits as intermediaries that manipulate data in some way. They are also very similar to the StreamProcessor representation defined by Ghani, Hancock and Pattinson.

With a few helper functions:

processorRead :: Monad m => (Maybe a -> Processor a m b) -> Processor a m b
processorRead k = construct (ProcessorRead k)

processorStop :: Monad m => Processor a m b
processorStop = construct ProcessorStop

processorEmit :: Monad m => b -> Processor a m b -> Processor a m b
processorEmit b proc = construct (ProcessorEmit b proc)

we can define a processor that filters:

filter :: Monad m => (a -> Maybe b) -> Processor a m b
filter h = processorRead getInput
  where
    getInput Nothing  = processorStop
    getInput (Just a) =
      case h a of
        Nothing -> filter h
        Just a  -> processorEmit a (filter h)

It is also possible to define several combinators that compose Streams with Processors, Processors with Readers and Processors with Processors in a similar manner to the Conduits library. We have same choices as with the composition of Streams and Readers in terms of the ordering of effects. The key point is that they may all be defined in terms of effectfulFold, and reasoned about using the universal property. For instance, we can prove that composition of Processors is associative, which opens the door to justified optimisation principles for chains of stream processors.

The Co-Inductive View

In the above I have taken an inductive view on Streams, Readers and so on. In Haskell, each recursive type is simultaneously an inductive and co-inductive type. We can use this change-of-viewpoint to give another way of defining interleaved data and effects in terms of unfolds. The following function unfold takes a seed state of type s and an evolution function of type s -> m (f s) and generates a value of type D m f.

2012-01-07 13:15 Was missing an additional Monad m constraint here.

unfold :: (Monad m, Functor f) => s -> (s -> m (f s)) -> D m f
unfold s step = loop s
  where loop s = D $ do
    f <- step s
    return (Step (fmap loop f))

The ofHandle stream defined above can be re-expressed in terms of unfold as follows:

ofHandle :: Handle -> Stream IO Char
ofHandle handle = unfold () step
  where
    step () = do
      isEOF <- hIsEOF handle
      if isEOF then do hClose handle
                       return StreamStop
               else do c <- hGetChar handle
                       return (StreamEmit c ())

In fact, we could change the representation of our interleaved data and effect type to be directly expressed in terms of unfolds:

data D m f where
    D :: s -> (s -> m (f s)) -> D m f

This representation makes it harder to define the effectfulFold function and its associated reasoning principle. On the other hand, it does allow for fusion of chained sequences of Streams, Processors and Readers in a similar manner to the Stream Fusion (alternative ACM paywall link) paper. Hopefully, this may allow for sequences of stream processing functions to be fused together and open up more possibilities for optimisation. But this remains to be seen.

A Type Checker that knows its Monad from its Elbow

Wed, 14 Dec 2011 00:00:00 +0000

I've been hacking a bit on Foveran lately. The main new thing that I've added is the integration of the monad laws and functor laws into the definitional equality. These additions were inspired by some suggestions of Conor, and a post on Epilogue by Pierre from some time back. I've taken a different implementation approach to the one sketched in that blog post. The scheme I am using is Filinski's normalisation by evaluation for the computational lambda-calculus, with some little extensions.

Descriptions of Indexed Functors form a (Relative) Monad

To describe (most of) its own data-types, Foveran implements the indexed descriptions (IDesc) data-type from the paper The Gentle Art of Levitation. Objects of type IDesc I are codes for functors from (I -> Set) to Set, where I is a Set. The IDesc I type has the following constructors:

“IId” : I -> IDesc I
“K”   : Set -> IDesc I
_“×”_ : IDesc I -> IDesc I -> IDesc I
“Sg”  : (A : Set) -> (A -> IDesc I) -> IDesc I
“Pi”  : (A : Set) -> (A -> IDesc I) -> IDesc I

The codes are all given their actual meanings by the semI operator I'll talk about below. But without really thinking about what the codes mean, one can easily see that there is a natural substitution operation on IDescs, simply because an object D : IDesc I is really just a tree with holes labelled with Is where the “IId”s are. Such a substitution operation would have type:

bind : (I J : Set) -> IDesc I -> (I -> IDesc J) -> IDesc J

This is pretty easy to implement: just recurse down the first IDesc I argument until you hit an “IId” i and then apply the i to the provided function. If I just implement bind, though, the type checker doesn't get to know that it satisfies the (relative) monad laws. (Side note: IDesc itself has type Set -> Set 1, so it is a relative monad).

So I've built-in the bind operation into the Foveran system and given it the following syntax:

bind x <- D1 in D2

where x is bound in D2.

When the Foveran type checker normalises terms of type IDesc I, it knows that they have one of two kinds of normal form: either they are constructed from one of the constructors above, and all the sub-IDesc Is are in normal form; or they are of the form bind x <- D1 in D2 where D1 is stuck and D2 is in normal form. By representing all objects of type IDesc I in this way, the normaliser is able to normalise terms up to the monad laws, as I'll now demonstrate.

If the normaliser is given a variable D : IDesc I with no definition, or some other stuck term, then its normal form is bind x <- D in “IId” x. So I automatically get the right-unit law for (relative) monads to hold definitionally in the Foveran type theory.

The internal implementation of the bind operator rearranges terms to always get things into the form described above. So nested bind applications get linearised:

  bind x <- (bind y <- D1 in D2) in D3
=
  bind y <- D1 in bind x <- D2 in D3

This implements the associativity laws of (relative) monads.

Finally bind knows what to do with any of the constructors above for building up IDesc I objects: it just commutes round them until it gets to an “IId” i, where it performs the actual substitution:

  bind x <- “IId” i in D
=
  D[i/x]

where D[i/x] means D with i substituted for x. So the left-unit law for (relative) monads holds definitionally. This is less interesting though, since this law always holds even if I define bind myself rather than building it into the type theory.

The Interpretation of Descriptions...

Given a object D of type IDesc I and a type X with a free variable i : I, the interpretation of D at X is given by the special type former semI[D, i. X], which obeys the following rules:

semI[“IId” e,   i. X] = X[e/i]
semI[“K” A,     i. X] = A
semI[D1 “×” D2. i. X] = semI[D1, i. X] × semI[D2, i. X]
semI[“Sg” A D,  i. X] = (a : A) × semI[D a, i. X]
semI[“Pi” A D,  i. X] = (a : A) -> semI[D a, i. X]

Given the description of the normal forms of the IDesc I that I described above, it would seem odd that there is no clause for semI on terms constructed from bind. So let us add one:

semI[bind x <- D1 in D2, i. X] = semI[D1, x. semI[D2, i. X]]

(remember that x may be free in D2). By integrating this law into the normaliser, I immediately get that the semantics of composed descriptions of functors is definitionally equal to the composition of the semantics of the descriptions. In short: the normaliser, and hence the definitional equality, knows that semI is a (relative) monad morphism!

... is a Functor

I've been saying that the IDesc type describes functors, but the semI operator only tells you what happens on objects. I could easily define a mapI function to implement an action on morphisms:

    mapI : (I : Set) ->
           (D : I -> IDesc I) ->
           (X Y : I -> Set) ->
           ((i : I) -> X i -> Y i) ->
           semI[D, i. X] -> semI[D, i. Y]

If I just implement this though, I am in the same position as with the bind function above: the definitional equality doesn't know that the functor laws hold. So I've added the following special operator to the Foveran type theory:

mapI[D, i. X, i. Y, f, x]

where D : IDesc I (the I gets inferred), X and Y are types each with a free I variable i, f : (i : I) -> X[i] -> Y[i], and x : semI[D, i. X]. The whole thing has type semI[D, i. Y]. (There's a lot more that could be inferred here, even without full-on type reconstruction, but this is what I've implemented at the moment).

Now I have to pull several tricks to get the functor laws to hold definitionally. The main one is forcing that the only normal forms of type semI[D, i. Y] when D is stuck are of the form mapI[D, i. X, i. Y, f, x].

So if it has a stuck term, like a variable x, of type semI[D, i. Y], the normaliser always expands it to a map of the (I-indexed) identity functor over x:

x = mapI[D, i. Y, i. Y, λi y. y, x]

This ensures that the identity preservation law always holds definitionally, similarly to the case for the right-unit law of the relative monad IDesc above.

As one might imagine, the rest of the implementation of mapI proceeds by recursion on the structure of the IDesc I argument, until it gets to a bind, whereupon it does something special:

2011-12-15 14:00: Thanks to Andrea Vezzosi in the comments for pointing out the mistake in the old version here. I also found a bigger mistake in the types. I think the new version is correct.

  mapI[bind x <- D1 in D2, i. Y, i. Z, f
      , mapI[D1, x. semI[D2 x, i. X], x. semI[D2 x, i. Y], g, z]]
=
  mapI[D1, x. semI[D2 x, i. X], x. semI[D2 x, i. Z]
      , λi x. mapI[D2 i, i. Y, i. Z, f, g i x]
      , z]

Note that the last argument on the top line must have this form, by the normal forms of stuck IDescs and the normal forms of values of type semI.

This rule ensures that mapI both satisfies the preservation of composition, and tracks the monad morphism behaviour of semI.

Making use: Inductive Types with Strictly Positive Parameters

There's an example of the use of the extended definitional equality in the file parameters.fv in the Foveran github repo. I'll give a quick overview below. Thanks to Stevan Andjelkovic for the idea.

Codes for Inductive Types with Parameters

The goal of the file is to give an account of inductive data types with parameters in such a way that I can define the functor laws generically (the IDesc codes above cannot, alas, define inductive data-types). For example, the type of lists of some type A uses the type A in a particular way that ensures that I can always define a generic map operation.

So I define a new type of codes of types indexed by some set I, with parameters indexed by some other set P:

IDescP : Set -> Set -> Set 1
IDescP P I = I -> IDesc (P + I)

To realise these codes as actual inductive types, I must compose them with something that handles the extra P + by instantiating it with the parameters:

IDescP:fork : (P I : Set) -> (P -> Set) -> P + I -> IDesc I
IDescP:fork P I X x =
  case x with { inl p. “K” (X p); inr i. “IId” i }

The complete codes are built using the following definition, which uses the bind operator to compose the descriptions:

IDescP:code : (P I : Set) -> IDescP P I -> (P -> Set) -> I -> IDesc I
IDescP:code P I D X =
  λi. bind x <- D i in IDescP:fork P I X x

Finally, they can be turned into actual I-indexed types, given the P-indexed family of parameters:

muP : (P I : Set) -> IDescP P I -> (P -> Set) -> I -> Set
muP P I D X = µI I (IDescP:code P I D X)

Defining Generic Map

The generic map function is now defined as follows (cata is the generically defined catamorphism function, derived from the built-in induction on inductive types):

map : (P I : Set) ->
      (D : IDescP P I) ->
      (X Y : P -> Set) ->
      ((p : P) -> X p -> Y p) ->
      (i : I) -> muP P I D X i -> muP P I D Y i
map P I D X Y f =
  cata I (IDescP:code P I D X)
     « muP P I D Y
     , λi x. construct
         (mapI[D i, x. semI[IDescP:fork P I X x, i. muP P I D Y i]
                  , x. semI[IDescP:fork P I Y x, i. muP P I D Y i]
                  , λx. case x with { inl p. f p; inr i. λx. x }
                  , x])
     »

This looks complicated, but this is mainly due to mapI not yet inferring as many of its arguments as it could. What is important is what I haven't written. In the λi x. construct... bit, the variable x has type semI[IDescP:code P I D X i, i. muP P I D Y i]. By the definition of IDescP:code this normalises to semI[D i, x. semI[IDescP:fork P I X x, i. muP P I D Y i] and I can easily apply mapI.

If the normaliser didn't know that semI was a relative monad morphism, then x's type would be semI[bind x <- D i in IDescP:fork P I X x, i. muP P I D Y i] and I would've had to manually decompose into the two parts, apply the mapI and then recompose. By making the definitional equality stronger, I can get away with writing less code.

What other Laws can be built-in?

What other laws can be built-in to the definitional equality? I believe that Conor wants to build the concept of free monad over a functor into Epigram, so that the monads laws automatically hold for all (for some intensionally defined version of "all") free monads. In the full-blown levitation scheme, IDesc would itself just be one free monad among many, and the laws would automatically hold for it. I think the semI and mapI laws would have to be still added specially though.

There are also subsets of the functors described by IDesc codes that have special properties. If one stays away from codes that introduce "data" (i.e. codes that describe containers with no shapes) then it is automatically an applicative functor. The lift/All construction used for describing generic induction is the leading example of this type. Dually, if one stays away from codes that introduce a choice of positions, one automatically get the (nameless?) dual of an applicative functor (e.g. the Somewhere modality). Maybe it would be interesting to let the relevant laws hold automatically in these cases as well?

How to be a Productive Programmer

Mon, 14 Nov 2011 00:00:00 +0000

On Friday, I gave a talk at the Scottish Programming Languages Seminar (SPLS) at Heriot-Watt. Many thanks to Greg Michaelson for organising everything and giving me time to speak.

I've put the slides I used on-line as a PDF file (with two small fixes, see below).

The talk presents an extension of a Nakano-style typed λ-calculus with a delay modality for guarded recursion. In short, this means:

The type system presented has a type-former ▷ so that for any type A, there is another type ▷ A representing an A delayed by one step in the current time stream.
Unlike Nakano's calculus, where the delay modality is presented using sub-typing rules, I have followed Conor's idea and presented the delay modality using applicative functor rules. I think this makes it a little easier to program with, and certainly easier to experiment with in an existing language like Haskell.
The really exciting new thing is the addition of clock variables, which allows for multiple time streams to be in play at once. This means it is possible to look at “all” of a potentially infinite computation and extract useful information from it. In the slides I presented the take and replaceMin functions as examples of this.

As I mentioned during the talk, this is based on Conor's blog post Time files like an Applicative Functor from over two years ago.

The hoped-for outcome is a nice way of programming with codata in a functional language like Haskell. Still more hopefully, this will extend to systems with dependently types too, though I am still confused about where the clock variables ought to live.

I fixed two typos that were present in copy of the slides that I used in the talk:

On slide 18 (page 42 in the PDF), I had “Time Files like an Applicative Functor” as the title of Conor's blog post. Obviously, time is rubbish at filing, or at least not as good as applicative functors are.
On slide 28 (page 57 in the PDF), I had the semantics of data type descriptions as an endofunction on some undefined curly D thing. It ought to be the powerset of the semantic domain.

On Structural Recursion II: Folds and Induction

Thu, 28 Apr 2011 00:00:00 +0000

Last time I talked about the background on defining structurally recursive functions in Type Theory and why you might want it. The key point is that structural recursion is driven by the data that is being analysed, as opposed to just doing its own thing with a side-proof that it always terminates.

The goal here is to come up with a self-contained and syntax-free definition of a dependently-typed structural recursion principle for a type A. To start with, this will be purely for normal structural recursion over inductively defined types. In future posts I will extend this to other types by combining existing structural induction principles.

In this post I'll cover how to present structural recursion and induction in a generic way. The first is probably well known to most people reading this, being just the standard stuff about initial F-algebras, but hopefully the second will be new.

The generic approach to structural induction I'll describe below underlies how Epigram 2 and my own language Foveran implement structural induction in a generic way. More information on Epigram's representation of data types is available in the paper The Gentle Art of Levitation.

My last post led to a very interesting discussion on reddit, which quickly left the measly content of my original behind. Just to clarify some of the points that came up in that discussion, I am only talking here about structural recursion/induction over inductively defined types---by which I mean types that arise as initial fixpoints of F-algebras---not co-inductive types (though the general technique for handling structural induction I'll describe below does extend to co-induction, at least in a categorical setting).

What is Structural Recursion?

Structural recursion is often presented as a restriction. In Coq or Agda, the programmer enters a recursive definition, and the system inspects it to make sure it fits the template of acceptable definitions. This "generate and test" approach naturally leads most people to think that the system is just being stupid when it refuses to accept their definition.

An alternative is for the system to be up-front about the definitions it is prepared to accept. Instead of writing recursive definitions and having them rejected, the programmer uses special-purpose recursion combinators to write their programs. This is the basic idea behind recursion schemes, as in the classic paper Functional Programming with Bananas, Lenses and Barbed Wire. An important extra benefit of using recursion schemes, as pointed out in that paper, is that they often also come with reasoning principles.

I am interested in going beyond normal recursion schemes, and doing dependent recursion schemes as well. This allows us to use structural recursion as a method of reasoning as well as a method of definition, and to intertwine the two if we feel like it. First though, I'll go over the standard stuff about how to treat the most basic recursion scheme at a general level.

Folds, Catamorphisms and F-Algebras

We can represent structural recursion on lists (with elements from a set A) using the right fold function:

foldr : (B : Set) → B → (A → B → B) → List → B

which satisfies the following two equations:

foldr B n c nil         = n
foldr B n c (cons a as) = c a (foldr B c n as)

As I mentioned last time, we can read these off directly as computation rules: when the implementation of foldr sees nil in its last argument, it knows to use its "nil" parameter; and when it sees cons a as in its third argument it knows to use the "cons" parameter. Thus we have a method for computation, driven by the structure of the data we are processing.

The well-known category-theoretic generalisation of this makes use of F-algebras. The basic idea is that the constructors of our inductive type are encoded as a functor F : Set → Set. For instance, the constructors of lists, with elements from some set A are encoded using the functor (I've only defined it here on objects, but the action on morphisms is obvious):

ListF : Set → Set
ListF X = 1 + A × X

The first two arguments of the foldr function can now be captured as a function f : ListF B → B. A pair of a set B and such a function f : F B → B is called an F-algebra. Given two F-algebras (k₁ : F B → B) and (k₂ : F C → C), we define F-algebra morphisms to be functions h : B → C such that h ○ k₁ = k₂ ○ F h. It is not too difficult to see that F-algebras and their morphisms form a category, called F-Alg.

For quite a large class of functors F, F-Alg has a (unique up-to isomorphism) initial object. Spelling this out, we have an F-algebra (in : F µF → µF) such that for any other F-algebra (k : F B → B) there is a unique F-algebra morphism from (in : F µF → µF) to (k : F B → B).

For the lists example, an initial algebra for the functor ListF is the set of lists with elements from the set A, with (in : ListF (List A) → List A) defined simply as the construction of lists using nil and cons. By initiality, if we have a ListF-algebra (k : ListF B → B) we get a function of type List A → B. Since it is a ListF-morphism it satisfies the two equations for foldr above, so we can interpret foldr.

More generally, we can interpret specifications of inductive types as functors F, the inductive types themselves as the carriers of initial algebras and structural recursion on them as making use of the initiality property to generate homomorphisms. Note that not every F you can write down necessarily has an initial algebra; for instance, F X = (X → 2) → 2 doesn't have an initial algebra in the category of sets and functions.

In the recursion schemes literature, folds are referred to as catamorphisms. Edward Kmett has written a comprehensive Knol on catamorphisms. He also has a field guide to recursion schemes, for both data and codata.

What is Structural Induction?

From a high level, (constructive) structural induction is just structural recursion with fancier types. And from a computation viewpoint this is fairly accurate: computation still proceeds by looking at data and then deciding what to do next. But we still need to work out what those types should be.

Dependent Structural Recursion (a.k.a. Structural Induction)

In pure Type Theory (i.e. without extensions for pattern matching and recursive definitions), structural recursion is presented in terms of elimination principles. Continuing the example of lists, we are provided with a pair of constructors nil : List and cons : A → List → List and an eliminator:

elimList : (P : List → Set) →
           (P nil) →
           ((a : A) → (l : List) → P l → P (cons a l)) →
           (l : List) → P l

This looks very similar to the type of the foldr function above. The difference is that the return type depends on the input list, and this dependency is carried through in the types of the "nil" and "cons" parameters (the second and third parameters).

As with the foldr function, elimList satisfies a pair of equations, which are almost identical, except for some extra information being passed in the cons case.

elimList P n c nil         = n
elimList P n c (cons a as) = c a as (elimList P n c as)

In presentations of type theory, the construction of elimination principles is usually defined syntactically by looking at the types of the constructors. See, for example, Dybjer's presentation of inductive families. This seems a little ad-hoc: one might hope for a generic way of extracting dependent eliminators for inductive types just from their presentations as initial F-algebras. Happily, this is possible.

Structural Induction for initial `F`-algebras

Given a functor F : Set → Set that has an initial algebra (in : F µF → µF) we first need to derive the type of the generic elimination principle for µF. The generic fold operator for µF has this type:

fold : (B : Set) → (F B → B) → µ F → B

Looking at the similarities between the foldr function on lists and the elimNat eliminator, we can guess that a generalisation should look like this for general (in : F µF → µF):

elimF : (P : µF → Set) →
        ((x : F µF) → lift F P x → P (in x)) →
        (x : µF) → P x

I'll attempt to explain this type with reference to the type of elimList. The first parameter, P : µF → Set is the dependent type we are eliminating to, and is exactly the same. The second parameter is the "algebra" component. It is a function that takes a value of type F µF. In the case of lists, this would either be inl * (representing nil) or inr (a,l) representing cons. So, similarly to F-algebras representing the multiple parameters to fold, we are encoding all the cases into one. The second argument of the algebra component is a little trickier. I have assumed a functor lift F : (µF → Set) → (F µF → Set) that takes P and x and produces some set. This part somehow encodes the "inductive hypothesis" for this structural recursion principle. I'll explain this further below. Finally, the "algebra" part ought to return a value of type P (in x), corresponding to the P nil and P (cons a l) parts of the elimList type.

Given such an "algebra" parameter, elimF promises to take any element x of µF and return a value of P x.

Lifting `F`

I need to give a definition for lift F : (F µF → Set) → (µ F → Set) now. In the case of lists, we can see that the following definition gives the right answer:

lift F P (inl *)     = ⊤
lift F P (inr (a,l)) = P l

Unfolding this definition in the type of elim above, it is easy to see that it is an encoding of the original type of elimList.

But how do we do this in general for any F? Hermida and Jacobs did this back in the nineties for polynomial F (i.e. Fs constructed from constants, sums, products and the identity functor) by induction on the structure:

lift Id      P x       = P x
lift (K A)   P a       = ⊤
lift (F × G) P (x,y)   = (lift F P x) × (lift F P y)
lift (F + G) P (inl x) = lift F P x
lift (F + G) P (inr y) = lift G P y

Note there are two cases for F + G depending on which component the (F + G) µH argument is in. It is easy to see that lift F is also a functor, so it has an action on morphisms between µF → Set objects.

Aside Note that this definition doesn't depend on the type of P being µ F → Set. We can actually give lift F the type (A → Set) → (F A → Set) for any A. So we can extend lift F to be a functor on the category Fam(Set), where objects are pairs (A : Set, P : A → Set). This extra generality helps when proving that structural induction is derivable from structural recursion, and for further applications like doing refinement of inductive types. Logical relations lovers will also see that this is the definition of the unary logical relation derived from the type scheme generated by a polynomial functor.

An important property of lift F is that it is truth-preserving. For any x : F µF, there is a (unique) value of lift F (λy. ⊤) x. I read this as saying that x : F µF completely determines the shape of lift F (λy. ⊤) x. We can use this to derive a useful function, using the functorality of lift F:

all : (P : µF → Set) →
      (x : F µF) →
      (k : (y : µF) → P y) →
      lift F P x

Knowing that lift F is truth preserving turns out to be just enough to prove that the elimF rule above holds, and also that the following equation holds:

elimF P h (in x) = h x (all P x (elimF P h))

Spelling this out for the case of lists, it is pretty easy to see that this gives us back the original elimination rule for lists. It is also true that elimF P h : (x : µF) → P x is the unique function that satisfies this equation, at least in the categorical setting. See Neil, Patty and Clem's paper for the category-theoretic proof that this works. The proof only demands that we have an initial algebra for F and a truth preserving lifting. As a hint of the proof, note that if we regard lift F as an endofunctor on Fam(Set) then the premises of the elimF function amount to taking a (lift F)-algebra with carrier (µF, P) to a morphism (x : µF) → K₁ µF x → P x, where K₁ µF x = ⊤. The proof consists of showing that K₁ µF is the carrier of the initial (lift F)-algebra.

Generalising to all `F`

I have only defined lift F for polynomial F so far, but it is possible to characterise lift F for any arbitrary F, as hinted by Hermida and Jacobs and fleshed out by Neil, Patty and Clem. Assuming we had extensional equality, we could define lift F as:

lift F P x = Σy : F (Σz : µF. P z). F π₁ y = x

where Σx : A. B x denotes a dependent pair type, and π₁ is the first projection. It is fairly easy to see that this is always truth-preserving, and that it coincides with the definition above for polynomial functors. This is particularly nice because it gives an extensional (i.e. we don't care about how F is constructed) definition of what the lifting is. It is also possible to give a generic description that works in any fibration with the right structure, so we can generalise to indexed types, types over categories of domains and so on.

When actually implementing this in a intensional type theory, like in Epigram 2 or Foveran, it is better, both because of the lack of extensional quality and for pragmatic reasons, to define the lifting directly in terms of the descriptions of the functors. However, the extensional definition does give a guide as to the form that lift F ought to take when we move beyond polynomial functors.

Can we generalise Structural Induction?

My goal in the next post is to show that structural induction on the structure of an inductively defined type is only one form of structural induction. Generalising from the set up above, my current thought for what a "structural induction principle" for some A : Set should look like consists of:

Another set B, which records the "sub-structure" of A
A constructor k : B → A
A predicate transformer H : (A → Set) → (B → Set)
A witness to truth-preservation tr : (b : B) → H (λa. ⊤) b
An eliminator elim : (P : A → Set) → ((b : B) → H P b → P (k b)) → (a : A) → P a

satisfying the equality

elim P h (k b) = h b (all P b (elim P h))

where all is derived from the witness to truth-preservation.

I should say that I'm not totally fixed on the details. My current thought is that I might need multiple constructors, or that more structure of H is required. But I'm pretty sure that an explicit description of how elements of the data type we are analysing are constructed is essential to any description of structural induction, because it allows us to state the computation rule. My plan is to push through examples and see what is required.

Previous work on this includes Nils Anders Danielsson's definition of Recursor in the Agda standard library, but this doesn't include a notion of constructor, or a computation rule. Conor also has a definition called Below which is pretty much identical.

Obviously structural induction on an inductively defined type fits into this schema, with B = F µF, and H = lift F. It is not so obvious that more general ones do. In the next few posts I'll cover how to generate new structural induction principles from old ones. For those of you that want to read ahead you can look at an outdated implementation, using an old definition of structural induction principle in the Foveran repository.

On Structural Recursion

Fri, 22 Apr 2011 00:00:00 +0000

This is the first in (hopefully) a series of blog posts on defining an algebra of structural induction principles in type theory, borrowing inspiration from category theory. This is joint work with Patty Johann and Neil Ghani, and builds on work and ideas of many, notably Conor McBride. In this post, I'll just explain the background.

Structural Recursion

Structural recursion is a fundamental part of the definition of functions in Type Theory, and also in functional programming languages. A standard example is that of length on lists (in Haskell syntax):

length : [a] -> Int
length []     = 0
length (x:xs) = 1 + length xs

Execution of length proceeds by looking at the constructor — either [] or _:_ — and then choosing a course of action. In the nil case, it evaluates to 0, in the cons case it recursively evaluates length, on the sub-term exposed by the pattern matching, and then adds 1 to it.

Definition by structural recursion has the following two features:

It is always terminating, because we only ever call the function again on smaller elements of the inductively defined type.
It provides an obvious way of computing. The process is: look at the constructor, do an action. We can define what it means to run functions, and hence use Type Theory as a programming language.

The first of these, while a nice to have feature in a functional programming setting, is essential in Type Theory, since we need totality of defined functions to retain decidability of type checking.

Nothing’s Ever Perfect

Definition by structural recursion is, at least on the surface, extremely restrictive. The restriction to recursive calls only on terms that are immediate sub-terms of the current argument is especially restrictive. The usual way to demonstrate this restrictiveness is the functional not-in-place "quicksort":

quicksort :: [a] -> [a]
quicksort []     = []
quicksort (x:xs) = quicksort l ++ quicksort (x:h)
  where (l,h) = partition x xs

For any reasonable definition of partition, the lists l and h will be sufficiently shorter than x:xs to ensure termination, but naïve structural recursion cannot see this.

To get around this kind of limitation, people have tried several different approaches.

General Recursion

Give up on termination and just use general recursion. This is the approach taken by most functional programming languages. General recursion can also be encoded in Type Theory if you have co-inductive types, as shown by Venanzio Capretta.

Termination Checkers

Instead of requiring that everything be defined using structural recursion, bolt another termination checker on to the type checker. This is the approach taken by Agda 2, and to a lesser extent by Coq.

This has the advantage that natural looking function definitions can be written, much as they would be written in a functional language with general recursion. The downside is that the termination checker is something separate to the core type theory, abrogating the claim of type theory to be a "checkable language of evidence" (I got this description from Conor McBride).

Structural Recursion on Accessibility Predicates

Exploit structural recursion by making use of a general purpose inductively defined type Acc that allows us to encode recursion on arbitrary well-founded relations (in Agda syntax):

data Acc (x : A) : Set where
    acc : (∀ y → y < x → Acc y) → Acc x

Functions are defined by structural recursion on a special Acc x argument. To actually call a function that requires such an argument we need to provide a proof that Acc x for the our chosen relation _<_ and our argument x. See Edward Z. Yang’s blog post for more information on doing this in Agda. As far as I am aware, this technique goes back to Bengt Nordström’s paper Terminating General Recursion.

In this technique, we are defining the function by structural recursion on the Acc x argument, so it is this argument that drives the computation. Since we probably want to erase this argument at run-time, this creates a gap between the semantics of functions at compile-time and functions at run-time.

A variant on this method is to generate a new inductively-defined predicate for each recursive function that you want to define. This is the Bove-Capretta method.

Can we regain a slavish devotion to structure?

In this series of blog posts, I’m going to explore a different technique that allows more complex structural recursion principles to be built up from basic structural recursion on inductively defined datatypes, in such a way that the data we are analysing still drives the computation. This idea goes back to (I think) Eduardo Giménez (apologies for the paywall link), who used it to justify Coq’s termination checker. The ideas were also promoted by McBride and McKinna’s The view from the left, where the systematic use of eliminators for defining functions in type theory was investigated. I’ll also be taking ideas from category-theoretic presentations of structural recursion, specifically Hermida and Jacobs’ original paper and Neil Ghani, Patricia Johann and Clement Fumex’s generalisation of it.

In the next post I’ll discuss how structural recursion is encoded in type theory and category theory.

Bob Atkey's blog

More Data Types with Negation at Fun in the REPL

Compiling higher-order specifications to SMT solvers

Simple semantics for defaults in Catala

Default Calculus

Catala values

Data types with Negation

Slides and Video for “Resource Constrained Programming with Full Dependent Types”

Quantitative Typing with Non-idempotent Intersection Types

Imports

Untyped λ-calculus

Krivine Abstract Machine

The Non-Idempotent Type Assignment System

Types, Observations, Behaviours

Typing contexts

Type Assignment

Measuring the sizes of derivations

Typing KAM configurations.

Quantitative Subject Reduction

Some lemmas

Subject Reduction

Subject expansion

Progress

Quantitative Soundness and Completeness

Slides for “Resource Constrained Programming with Full Dependent Types”

Off the Beaten Track 2017: Call for Talk Proposals

Background

Scope

Previous OBTs

Important Dates

Submission

Organisers

Authenticated Data Structures, as a Library, for Free!

Authenticated Data Structures as a Library

An Autheticated Data Structure: Merkle Trees

The Prover

Proofs

Prover Implementation

Trying out the Prover

The Verifier

Trying out the Verifier

Attempting to trick the verifier

Correctness from Parametricity

Slides and notes for my OBT “Generalising Abstraction” talk

The Incremental λ-Calculus and Relational Parametricity

A Theory of Changes, and Change Structures

Relational Parametricity via Reflexive Graphs

Change Structures and Reflexive Graphs are equivalent

So what?

Slides for “An Algebraic Approach to Typechecking and Elaboration”

Propositions as Filenames, Builds as Proofs: The Essence of Make

The Essence of make

What make does

So what?

POPL Slides

From Parametricity to Conservation Laws, via Noether's Theorem

A Relationally Parametric Model of Dependent Type Theory

One Done, Two Submitted

Productive Coprogramming

Pair of Papers Pertaining to Parametricity

Dependent Types

Classical Mechanics

Productive Coprogramming with Guarded Recursion

Abstraction and Invariance for Algebraically Indexed Types

Theorems for Free

Interleaving Data and Effects

Relational Parametricity for Higher Kinds

Reasoning about Stream Processing with Effects

Effectful Streams

Stream Readers

Generic Interleaved Data and A Recursion Scheme

Eilenberg-Moore Algebras

A Recursion Scheme

Using the Recursion Scheme

Back to Readers

Processors

The Co-Inductive View

A Type Checker that knows its Monad from its Elbow

Descriptions of Indexed Functors form a (Relative) Monad

The Interpretation of Descriptions...

The Essence of `make`

What `make` does

Structural Induction for initial `F`-algebras

Lifting `F`

Generalising to all `F`