Draft Proposal for SES (Secure EcmaScript)

  • Most of the action on the SES-shim is happening at https://github.com/Agoric/SES-shim/tree/master/packages/ses .
  • Moddable is also directly building a SES machine as the primary configuration of the XS implementation of JS for embedded, as standardized in Ecma TC53.
  • The most relevant proposal these days is https://github.com/tc39/proposal-compartments which is much more up to date than this proposal repository is.
  • We should update this proposal repository with text from https://github.com/Agoric/SES-shim/tree/master/packages/ses which is much more current.
  • MetaMask and Agoric both have plugin architectures running on SES.
  • MetaMask and MetaMask's LavaMoat run on the SES-shim.
  • The Agoric framework runs on both on the SES-shim on Node, and increasingly on a version of XS called XSnap that also supports orthogonal persistence.

Note that this proposal was previously called "proposal-frozen-realms". However, with progress on proposal-realms, the realms-shim, and the ses-shim, we found we no longer needed to distinguish frozen-realms from SES. Most historical references to "Frozen Realms" are best interpreted as being about an older version of SES.

Champions

  • Mark S. Miller, Agoric
  • JF Paradis, Agoric
  • Caridy Patiño, Salesforce
  • Patrick Soquet, Moddable
  • Bradley Farias, GoDaddy, Node

This document specifies "compartments", a concept focused on making lightweight realms designed to be used with a shared immutable realm. The proposal here is intended to compose well with the various Realm proposals but is independent. These proposals each have utility without the other, and so can be proposed separately. However, together they have more power than each separately.

We motivate the SES API presented here with a variety of examples.

Status

Current Stage:

  • Stage 1

Moddable's Compartment API, the direct ancestor to this proposal, implemented on the XS SES engine.

Making Javascript Safe and Secure Talks by Mark S. Miller (Agoric), Peter Hoddie (Moddable), and Dan Finlay (MetaMask)

Presentation to TC53 Omit Needless Words

LavaMoat - Securing your dependency graph by Kumavis (MetaMask)

Presentation to Node Security Securing EcmaScript

Historical

Automated Analysis of Security-Critical JavaScript APIs by Ankur Taly Úlfar Erlingsson John C. Mitchell Mark S. Miller Jasvir Nagra

Frozen Realms: Draft Standard Support for Safer JavaScript Plugins is an in-depth talk that covers the important ideas, but is very stale regarding specifics.

The old Realms API proposal and the current Realms proposal. The original plan was to settle the Realms proposal first, but with the current approach, this is no longer required.

The original efforts to rebuild frozen realms on top of these Realms is:

Summary

In ECMAScript, a realm consists of a global object and an associated set of primordial objects -- mutable objects like Array.prototype that must exist before any code runs. Objects within a realm implicitly share these primordials and can therefore easily disrupt each other by primordial poisoning -- modifying these objects to behave badly. This disruption may happen accidentally or maliciously. Today, in the browser, realms can be created via same origin iframes, and in Node via vm contexts. On creation, these realms are separate from each other because they share no mutable state. Because prototypes are mutable, each realm needs its own set, making this separation too expensive to be used at fine grain.

Realms are currently not exposed directly to JavaScript but are represented in the specs by the realm record, of which the most important slots are the intrinsics, the global object, and the global lexical environment (see ECMA262 sections 8.2 Realms).

We propose to add the concept of compartments, to designate lightweight child realms inside a realm. Each compartment has its own global object and global lexical scope, but all compartments inside a given realm share their intrinsics. Separation is achieved by making the intrinsics immutable, preventing an object in one compartment from poisoning the prototypes used by the other compartments.

This means that each compartment consists of a new global object, and a new global lexical environment:

Record slots Realm Compartment
intrinsics mutable immutable, shared
global object mutable mutable
global lexical scope mutable mutable

The compartment record is like a realm record, except that its intrinsics slot points to the parent realm record. Everywhere the specs refers to the the realm record, the compartment record can be subsituted with no further changes.

The Compartment constructor

Superceded by the Compartments proposal.

We propose a Compartment class, whose instances is a reification of the concept of "compartment" introduced above, for making multiple lightweight child realms inside a given realm.

Though initially separate, compartments can be brought into intimate contact with each other via global object and modules.

class Compartment {
  constructor: (
    endowments: object?,     // extra bindings added to the global object
    moduleMap: object?,      // maps child specifier to parent specifier
    options: object?         // including hooks like isDirectEvalHook
  ) -> object                // an exotic compartment object

  get global -> object       // access this compartment's global object

  evaluate(                  // do a strict indirect eval in this compartment
    src: stringable,
    options: object?         // per-evaluation rather than per-compartment
  ) -> any

  // same signature as dynamic import
  async import(specifier: string) -> promise<ModuleNamespace>
  importSync(specifier: string) -> ModuleNamespace
}

The compartment constructor creates a new lightweight child realm with a new global, a new eval function, a new Function constructor, and a new Compartment constructor.

  • The compartment global object consists of all the primordial state defined by ECMA262, but contains no host provided objects, so window, document, XMLHttpRequest, require, process etc. are all absent. Thus, a compartment contains none of the objects needed for interacting with the outside world, like the user or the network.

  • The new eval, Function, and Compartment will evaluate code in the global scope of the new compartment: the new compartment's global becomes their global object.

  • The new eval, Function, and Compartment inherit from the shared %FunctionPrototype%.

  • The new Function.prototype is the shared %FunctionPrototype%.

  • The new Compartment constructor...?

  • The new Compartment.prototype is the shared %CompartmentPrototype%.

The constructor then copies the values of the own enumerable properties from the endowments parameter onto the new global and returns the new compartment instance. With these additional endowments, users provide the virtual host objects that they wish to be available in the spawned compartment.

The Compartment constructor is only available on the global object after lockdown has been invoked (see below).

The Compartment prototype

We propose on the shared Compartment.prototype, to be inherited by instances of the all Compartment classes:

  • a global getter to provide access to the compartment global object. Its behavior is similar to the globalThis global object.
  • an evaluate method to evaluate code in the global scope of the new compartment. Its signature is identical to the eval() function but possibly with an additional optional options argument.
  • an asynchronous import method to dynamically load modules in the new compartment. Its signature is identical to the dynamic import function.

The lockdown method

We propose a static method, lockdown() or Realm.lockdown(), for converting the current realm into a state with immutable primordials. We call such a realm an immutable realm. The Realm global object will be specified by the Realms proposal.

The lockdown operation consists of:

  • taming some globals (see below).
  • taming the function constructors (see below).
  • freezing all intrinsics (see below).
  • disabling the default mechanism causing the override mistake (see below).
  • exposing the Compartment constructor via the global object which is not available before lockdown (see below).

Although Compartment and Realm.lockdown() appear orthogonal, they are only interesting when directly composed:

Realm.lockdown();
const cmpA = new Compartment();
const cmpB = new Compartment();

After lockdown, all the primordials that cmpA and cmpB share are immutable, so neither can poison the prototypes of the other. Because they share no mutable state, they are as fully separate from each other as two full realms created by two same origin iframes (except the shared identity of frozen primordials, thus avoiding identity discontinuity explained below).

Modification of the prototypes is allowed before lockdown is called (which raises interesting issues re what is frozen by lockdown).

(edit with next two paragraphs)

A long recognized best practice is "don't monkey-patch primordials" -- don't mutate any primordial state. Most legacy code obeying this practice is already compatible with lightweight realms descending from an immutable root realm. Some further qualifications are explained in the rest of this document.

If customization of the intrinsics is required, it can be done before lockdown is called and before any compartment is created.

The Compartment global object

The compartment constructor is unavailable before lockdown() is called, to avoid the risk of omitting lockdown and creating compartments with non-frozen primordials (which would not provide the intended isolation).

Freezing intrinsics and Taming globals

In order for the intrinsics to be shared safely, they must be transitively immutable. Fortunately, of the standard primordials in ES2016, the only mutable primordial state is:

  • Mutable own properties of primordial objects
  • The mutable internal [[Prototype]] slot of primordial objects
  • The ability to add properties
  • Math.random
  • Date.now
  • The Date constructor called as a constructor with no arguments
  • The Date constructor called as a function, no matter what arguments (Surprised me!)
  • Normative optional proposed RegExp static methods (link)
  • Normative optional proposed Error.prototype.stack accessor (link)

To make a transitively immutable root realm, we, respectively

  • Remove all non-standard properties
  • Remove Math.random
  • Remove Date.now
  • Have new Date() throw a TypeError
  • Have Date(any...) throw a TypeError
  • Remove the RegExp static methods if present
  • Remove RegExp.prototype.compile
  • Remove Error.prototype.stack if present
  • Make all primordial objects non-extensible.
  • Make all remaining properties non-configurable, non-writable. If an accessor property, we specify that its getter must always return the same value without mutating any state, and its setter either be absent or throw an error without mutating any state.

Likewise, any new addition to the specifications need to follow the same policy, in order to avoid introducing mutable state in a compartment.

A user can effectively add the missing functionality of Date and Math back in when necessary, or substitute safe implementations. For example

const DateNow = Date.now;

Realm.lockdown();

function unsafeDate() {
  return Date(...arguments);
}
Object.defineProperties(unsafeDate, Object.getOwnPropertyDescriptors(Date));
Object.defineProperty(unsafeDate, 'now', {
    value: DateNow,
    writable: true,
    enumerable: false,
    configurable: true
});

const cmp = new Compartment({ Date: unsafeDate });

Taming the function constructors.

All intrinsics are shared, but the %Function%, %GeneratorFunction%, %AsyncFunction% and %AsyncGeneratorFunction% perform by default source code evaluation in the global scope of the realm.

After lockdown, these constructor should be replaced with functions that throw instead of evaluating source code, so they can be safely shared. We could specify that their throwing behavior is the same as when the host hook (for CSP) suppresses evaluation, mapping it to an already possible behavior. If Compartment is a per-realm global rather than per-Compartment, then Compartment.prototype.constructor === Compartment, which is not tamed? Let's talk about this.

Override mistake

Because of lack of sufficient foresight at the time, ES5 unfortunately specified that a simple assignment to a non-existent property must fail if it would override a non-writable data property of the same name. (In retrospect, this was a mistake, but it is now too late and we must live with the consequences.) It is inconsistent with overriding by classes and object literals, since they do [[DefineOwnProperty]] rather than assignment.

As a result, simply freezing an object to make it immutable has the unfortunate side effect of breaking previously correct code that is considered to have followed JS best practices, if this previous code used assignment to override. For example this assignment will fail:

Object.freeze(Array.prototype);
const arr = []
arr.join = true; // throws in strict mode, ignore in sloppy mode.

For that reason, after freezing the primordials, we need to Make non-writable prototype properties not prevent assigning to instance.

See the override mistake. (better link?)

(We need another bit of semantic state to distinguish these two ways of being frozen. We should specify that petrify and perhaps even harden also protect against override mistake, even though we avoid fully shimming that.)

Identity discontinuity

Two realms, made by same origin iframes or vm contexts, can be put in contact. Once in contact, they can mix their object graphs freely. When realms do this, they encounter an inconvenience and source of bugs we will here call identity discontinuity. For example if code from iframeA makes an array arr that it passes to code from iframeB, and iframeB tests arr instanceof Array, the answer will be false since arr inherits from the Array.prototype of iframeA which is a different object than the Array.prototype of iframeB.

By contrast, since cmpA and cmpB share the same Array.prototype, an array arr created by one still passes the arr instanceof Array as tested by the other.

###################################

TODO BELOW

Confinement examples

function confine(src, endowments) {
  return sharedRoot.spawn(endowments).eval(src);
}

This confine function is an example of a security abstraction we can easily build by composing the primitives above. It uses spawn to make a lightweight realm descendant from our immutable sharedRoot realm above, copies the own enumerable properties of endowments onto the global of that lightweight realm, and then evaluates src in the scope of that global and returns the result. This confine function is especially useful for object-capability programming. These primitives (together with membranes) can also help to support other security models such as decentralized dynamic information flow though more mechanism may additionally be needed. We have not yet explored this in any detail.

(The confine function is from SES, which has a formal semantics supporting automated verification of some security properties of SES code. It was developed as part of the Google Caja project; you can read more about SES and Caja on the Caja website.)

confine('x + y', {x: 3, y: 4})  // -> 7

confine('Object', {})           // -> Object constructor of an immutable root

confine('window', {})           // ReferenceError, no 'window' in scope

Plugin separation example

function Counter() {
  let count = 0;
  return Object.freeze({
    incr: Object.freeze(() => ++count),
    decr: Object.freeze(() => --count)
  });
}
const counter = new Counter();

// ...obtain billSrc and joanSrc from untrusted clients...
const bill = confine(billSrc, {change: counter.incr});
const joan = confine(joanSrc, {change: counter.decr});

Say the code above is executed by a program we call Alice. Within this code, Alice obtains source code for plugins Bill and Joan. Alice does not know how well these plugins are written, and so wishes to protect herself from their misbehavior, as well as protect each of them from the misbehavior of the other. It does not matter whether Alice is worried about accidental or malicious misbehavior.

With the code above, Alice presents to each of these plugins an API surface of her design, characteristic of the plugin framework she defines. In this trivial example, she provides to each a function they will know as change for manipulating the state of a shared counter. By calling his change variable, Bill can only increment the counter and see the result. By calling her change variable, Joan can only decrement the counter and see the result. By using her counter variable Alice can do both.

If Alice's code above is normal JavaScript code, then she does not achieve this goal. For example, Bill or Joan could use the expression change.__proto__ to access and poison Alice's prototypes, and to interact with each other in ways Alice did not intend to enable. The API surface that Alice exposed to Bill and Joan was not defensive; it did not protect itself and Alice from Bill and Joan's misbehavior.

Alice's code above is properly defensive if it is evaluated in a realm descendant from an immutable root realm. Alice places Bill and Joan in such a realm to confine them. She places herself in such a realm for its defensibility, which Alice can use to define defensive abstractions that are safe to expose to Bill and Joan. If Alice, Bill, and Joan all descend from sharedRoot, then their further interactions are defensible and free of identity discontinuities.

A convenience: def(obj)

All those calls to Object.freeze above are ugly. The Caja def(obj) function is an example of a convenience that should be provided by a library. It applies Object.freeze recursively to all objects it finds starting at obj by following property and [[Prototype]] links. This gives all these objects a tamper proof API surface (Note, though, that it does not make them immutable except in special cases.) The name def means "define a defensible object".

Using def, we can rewrite our Counter example code as

function Counter() {
  let count = 0;
  return def({
    incr() { return ++count; }
    decr() { return --count; }
  });
}

To be efficient, def needs to somehow be in bed with this proposal, so it can know to stop traversing when it hits any of these transitively immutable primordials. We leave it to a later proposal to work out this integration issue.

Compartments example

By composing revocable membranes and confine, we can make compartments:

function makeCompartment(src, endowments) {
  const {wrapper,
         revoke} = makeMembrane(confine);
  return {wrapper: wrapper(src, endowments),
          revoke};
}

// ...obtain billSrc and joanSrc from untrusted clients...
const {wrapper: bill,
       revoke: killBill} = makeCompartment(billSrc, endowments);
const {wrapper: joan,
       revoke: killJoan} = makeCompartment(joanSrc, endowments);

// ... introduce mutually suspicious Bill and Joan to each other...
// ... use both ...
killBill();
// ... Bill is inaccessible to us and to Joan. GC can collect Bill ...

After killBill is called, there is nothing the Bill code can do to cause further effects.

Detailed Proposal

You can view the spec text draft in ecmarkup format or rendered as HTML.

  1. Introduce the Realm class as an officially recognized part of the ECMAScript standard API.

  2. Add to the Realm class a static method, Realm.immutableRoot(), which obtains an immutable root realm in which all primordials are already transitively immutable. These primordials include all the primordials defined as mandatory in ES2016. (And those in draft ES2017 as of March 17, 2016, the time of this writing.) These primordials must include no other objects or properties beyond those specified here. In an immutable root realm the global object itself is also transitively immutable. Specifically, it contains no host-specific objects. This frozen global object is a plain object whose [[Prototype]] is Object.prototype, i.e., the %ObjectPrototype% intrinsic of that immutable root realm.

    • Since two immutable root realms are forever the same in all ways except object identity, we leave it implementation-defined whether Realm.immutableRoot() always creates a fresh one, or always returns the same one. On any given implementation, it must either be always fresh or always the same.

  3. In order to attain the necessary deep immutability of an immutable root realm, two of its primordials must be modified from the existing standard: An immutable root realm's Date object has its now() method removed and its default constructor changed to throw a TypeError rather than reveal the current time. An immutable root realm's Math object has its random() method removed.

  4. Add to the Realm class an instance method, spawn(endowments).

    1. spawn creates a new lightweight child realm with its own fresh global object (denoted below by the symbol freshGlobal) whose [[Prototype]] is the parent realm's global object. This fresh global is also a plain object. Unlike the global of an immutable root realm, this new freshGlobal is not frozen by default.

    2. spawn populates this freshGlobal with overriding bindings for the evaluators that have global names (currently only eval and Function). It binds each of these names to fresh objects whose [[Prototype]]s are the corresponding objects from the parent realm.

    3. spawn copies the own enumerable properties from the endowments record onto the freshGlobal.

    4. spawn returns that new child realm instance.

    The total cost of a lightweight realm is four objects: the realm instance itself, the freshGlobal, and the eval function and Function constructor specific to it.

  5. The evaluators of a spawned realm evaluate code in the global scope of that realm's global, using that global as their global object.

    A lightweight realm's initial eval inherits from its parent's eval. For each of the overriding constructors (currently only Function), its prototype property initially has the same value as the constructor they inherit from. Thus, a function foo from one descendant realm passes the foo instanceof Function test using the Function constructor of another descendant of the same parent realm. Among sibling lightweight realms, instanceof on primordial types simply works.

Polyfill example

In the Punchlines section below, we explain the non-overt channel threats that motivate the removal of Date.now and Math.random. However, usually this threat is not of interest, in which case we'd rather include the full API of ES2016, since it is otherwise safe. Indeed, Caja has always provided the full functionality of Date and Math because Caja's threat model did not demand that they be denied.

The following makeColdRealm(GoodDate, goodRandom) function, given a good Date constructor and Math.random function, makes a new frozen-enough lightweight realm, that can be used as if it is an immutable root realm -- as a spawning root for making lightweight child realms. These children are separated-enough from each other, if one is not worried about non-overt channels. Unlike the lightweight realms directly descendant from an immutable root realm, children spawned from a common cold realm share a fully functional Date and Math.

function makeColdRealm(GoodDate, goodRandom) {
  const goodNow = GoodDate.now;
  const {Date: SharedDate, Math: SharedMath} = sharedRoot;
  function FreshDate(...args) {
    if (new.target) {
      if (args.length === 0) {
        args = [+goodNow()];
      }
      return Reflect.construct(SharedDate, args, new.target);
    } else {
      return String(GoodDate());
    }
  }
  FreshDate.now = () => +goodNow();
  FreshDate.prototype = SharedDate.prototype;  // so instanceof works
  FreshDate.name = SharedDate.name;
  FreshDate.__proto__ = SharedDate;

  const FreshMath = {
    __proto__: SharedMath,
    random() { return +goodRandom(); }
  };

  return def(sharedRoot.spawn({Date: FreshDate, Math: FreshMath}));
}

In addition to Date and Math, we can create abstractions to endow a fresh global with virtualized emulations of expected host-provided globals like window, document, or XMLHttpRequest. These emulations may map into the caller's own or not. Caja's Domado library uses exactly this technique to emulate most of the conventional browser and DOM APIs, by mapping the confined code's virtual DOM into the portions of the caller's "physical" DOM that the caller specifies. In this sense, the confined code is like user-mode code in an operating system, whose virtual memory accesses are mapped to physical memory by a mapping it does not see or control. Domado remaps URI space in a similar manner. By emulating the browser API, much existing browser code runs compatibly in a virtualized browser environment as configured by the caller using Domado.

Because eval, Function, and the above Date and Math observably shadow the corresponding objects from their parent realm, the spawned environment is not a fully faithful emulation of standard ECMAScript. However, these breaks in the illusion are a necessary price of avoiding identity discontinuities between lightweight realms spawned from a common parent. We have chosen these breaks carefully to be compatible with virtually all code not written specifically to test standards conformance.

Mobile code example

Map-Reduce frameworks vividly demonstrate the power of sending the code to the data, rather than the data to the code. Flexible distributed computing systems must be able to express both.

Now that Function.prototype.toString will give a reliably evaluable string that can be sent, an immutable root realm provides a safe way for the receiver to evaluate it, in order to reconstitute that function's call behavior in a safe manner. Say we have a RemotePromise constructor that makes a remote promise for an object that is elsewhere, potentially on another machine. Below, assume that the RemotePromise constructor initializes this remote promise's private instance variable #farEval to be another remote promise, for the eval method of an immutable root realm at the location (vat, worker, agent, event loop, place, ...) where this promise's fulfillment will be. If this promise rejects, then its #farEval promise likewise rejects.

class QPromise extends Promise {
  // ... API from https://github.com/kriskowal/q/wiki/API-Reference
  // All we actually use below is fcall
}

// See https://github.com/kriskowal/q-connection
class RemotePromise extends QPromise {
  ...
  // callback must be a closed function, i.e., one whose only free
  // variables are the globals defined by ES2016 and therefore present
  // on the proto-global.
  there(callback, errback = void 0) {
    const callbackSrc = Function.prototype.toString.call(callback);
    const farCallback = #farEval.fcall(callbackSrc);
    return farCallback.fcall(this).catch(errback);
  }
}

We explain there by analogy. The familiar expression Promise.resolve(p).then(callback) postpones the callback function to some future time after the promise p has been fulfilled. In like manner, the expression RemotePromise.resolve(r).there(callback) postpones and migrates the closed callback function to some future time and space, where the object that will be designated by the fulfilled remote promise r is located. Both then and there return a promise for what callback or errback will return.

This supports a federated form of the Asynchronous Partitioned Global Address Space concurrency model used by the X10 supercomputer language, integrated smoothly with our promise framework for handling asynchrony.

How Deterministic?

We do not include any form of replay within the goals of this proposal, so this "How Deterministic" section is only important because of the punchlines at the end of this section.

Given a deterministic spec, one could be sure that two computations, run on two conforming implementations, starting from the same state and fed the same inputs, will compute the same new states and outputs. The ES5 and ES2015 specs come tantalizingly close to being deterministic. ECMAScript has avoided some common but unnecessary sources of non-determinism like Java's System.identityHashCode or the enumeration order of identity hash tables. But the ECMAScript specs fail for three reasons:

  • Genuine non-determinism, such as by Math.random().
  • Unspecified but unavoidable failure, such as out-of-memory.
  • Explicit underspecification, i.e. leaving some observable behavior up to the implementation.

The explicitly non-deterministic abilities to sense the current time (via new Date() and Date.now()) or generate random numbers (via Math.random()) are disabled in an immutable root realm, and therefore by default in each realm spawned from it. New sources of non-determinism, like makeWeakRef and getStack will not be added to immutable root realms or will be similarly disabled.

The ECMAScript specs to date have never admitted the possibility of failures such as out-of-memory. In theory this means that a conforming ECMAScript implementation requires an infinite memory machine. Unfortunately, such machines are currently difficult to obtain. Since ECMAScript is an implicitly-allocating language, the out-of-memory condition could cause computation to fail at any time. If these failures are reported by unpredictably throwing a catchable exception, then defensive programming becomes impossible. This would be contrary to the goals of much ECMAScript code. Thus, any ECMAScript computation that wishes to defend its invariants, and any synchronous computation it is entangled with must, on encountering an unpredictable error, preemptively abort without running further user code.

Even if ECMAScript were otherwise deterministically replayable, these unpredictable preemptive failures would prevent it. We examine instead the weaker property of fail-stop determinism, where each replica either fails, or succeeds in a manner identical to every other non-failing replica.

Although few in number, there are specification issues that are observably left to implementations, upon which implementations may differ. Some of these may eventually be closed by future TC39 agreement, such as enumeration order if objects are modified during enumeration (TODO link). Others, like the sort algorithm used by Array.prototype.sort are less likely to be closed. However, implementation-defined is not necessarily genuine non-determinism. On a given implementation, operations which are only implementation-defined can be deterministic within the scope of that implementation. They should be fail-stop reproducible when run on the same implementation. To make use of this for replay, however, we would need to pin down what we mean by "same implementation", which seems slippery and difficult.

The punchlines

Even without pinning down the precise meaning of "implementation-defined", a computation that is limited to fail-stop implementation-defined determinism cannot read covert channels and side channels that it was not explicitly enabled to read. Nothing can practically prevent signaling on covert channels and side channels, but approximations to determinism can practically prevent confined computations from perceiving these signals.

(TODO explain the anthropic side channel and how it differs from an information-flow termination channel.)

Fail-stop implementation-defined determinism is a great boon to testing and debugging. All non-deterministic dependencies, like the allegedly current time, can be mocked and injected in a reproducible manner.

Annex B considerations

As of ES2016, the normative optionals of Annex B are safe for inclusion as normative optionals of immutable root realms. However, where Annex B states that these are normative mandatory in a web browser, there is no such requirement for immutable root realms. Even when run in a web browser, an immutable root realm, having no host specific globals, must be considered a non-browser environment. Some post-ES2015 APIs proposed for Annex B, such as the RegExp statics and the Error.prototype.stack accessor property, are not safe for inclusion in immutable root realms and must be absent.

At this time, to maximize compatibility with normal ECMAScript, we do not alter an immutable root realm's evaluators to evaluate code in strict mode by default. However, we should consider doing so. Most of the code, including legacy code, that one would wish to run under an immutable root realm is probably already compatible with strict mode. Omitting sloppy mode from immutable root realms and their spawned descendants would also make sections B.1.1, B.1.2, B.3.2, B.3.3, and B.3.4 non issues. It is unclear what an immutable root realm's evaluators should specify regarding the remaining normative optional syntax in section B.1. But the syntax accepted by these evaluators, at least in strict mode, should probably be pinned down precisely by the spec.

Some of the elements of Annex B are safe and likely mandatory in practice, independent of host environment:

  • escape and unescape
  • Object.prototype.__proto__
  • String.prototype.substr
  • The String.prototype methods defined in terms of the internal CreateHTML: anchor, big, ..., sup
  • Date.prototype.getYear and Date.prototype.setYear
  • Date.prototype.toGMTString
  • __proto__ Property Names in Object Initializers

All but the last of these have been whitelisted in Caja's SES-shim for a long time without problem. (The last bullet above is syntax and so not subject to the SES-shim whitelisting mechanism.)

Discussion

Because an immutable root realm is transitively immutable, we can safely share it between ECMAScript programs that are otherwise fully isolated. This sharing gives them access to shared objects and shared identities, but no ability to communicate with each other or to affect any state outside themselves. We can even share immutable root realms between origins and between threads, since deep immutability at the specification level should make thread safety at the implementation level straightforward.

Today, to self-host builtins by writing them in ECMAScript, one must practice safe meta programming techniques so that these builtins are properly defensive. This technique is difficult to get right, especially if such self hosting is opened to ECMAScript embedders. Instead, these builtins could be defined in a lightweight realm spawned from an immutable root realm, making defensiveness easier to achieve with higher confidence.

By the rules above, a spawned realm's Function.prototype.constructor will be the parent realm's Function constructor, i.e., identical to the spawned realm's Function.__proto__. In exchange for this odd topology, we obtain the pleasant property that instanceof works transparently between spawned realms by default -- unless overridden by a user's polyfill to the contrary.

In ES2016, the GeneratorFunction evaluator is not a named global, but rather an unnamed intrinsic. Upcoming evaluators are likely to include AsyncFunction and AsyncGeneratorFunction. These are likely to be specified as unnamed intrinsics as well. For all of these, the above name-based overriding of spawn is irrelevant and probably not needed anyway.

Because code evaluated within an immutable root realm is unable to cause any affects outside itself it is not given explicit access to, the evaluators of an immutable root realm should continue to operate even in environments in which CSP has forbidden normal evaluators. By analogy, CSP evaluator suppression does not suppress JSON.parse. There are few ways in which evaluating code in an immutable root realm is more dangerous than JSON data.

Other possible proposals, like private state and defensible const classes, are likely to aid the defensive programming that is especially powerful in the context of this proposal. But because the utility of such defensive programming support is not limited to frozen realms, they should remain independent proposals.

For each of the upcoming proposed standard APIs that are inherently not immutable and powerless:

they must be absent from an immutable root realm, or have their behavior grossly truncated into something safe. This spec will additionally need to say how they initially appear, if at all, in each individual spawned lightweight realm. In particular, we expect a pattern to emerge for creating a fresh loader instance to be the default loader of a fresh spawned realm. Once some proposed APIs are specced as being provided by import from builtin primordial modules, we will need to explain how they appear in an immutable root realm and/or the realms it spawns.

Open Questions

  • Should Realm.immutableRoot() return a new fresh frozen realm each time or should it always return the same one? Above we leave this implementation-defined for now to encourage implementations to experiment and see how efficient each can be made. If all can agree on one of these options, we should codify that rather than continue to leave this implementation-defined.

  • Although not officially a question within the jurisdiction of TC39, we should discuss whether the existing CSP "no script evaluation" settings should exempt an immutable root realm's evaluators, or whether CSP should be extended in order to express this differential prohibition.

  • Currently, if the value of eval is anything other than the original value of eval, any use of it in the form of a direct-eval expression will actually have the semantics of an indirect eval, i.e., a simple function call to the current value of eval. If an immutable root realm's builtin evaluators are not strict by default, then any user customization that replaces a spawned realm's global evaluators with strict-by-default wrappers will break their use for direct-eval. Fortunately, this seems to be addressed by the rest of the old Realms API.

  • The standard Date constructor reveals the current time either

    • when called as a constructor with no arguments, or
    • when called as a function (regardless of the arguments)

    Above we propose to censor the current time by having the proto-Date constructor throw a TypeError in those cases. Would another error type be more appropriate? Instead of throwing an Error, should new Date() produce an invalid date, equivalent to that produced by new Date(NaN)? If so, calling the Date constructor as a function should produce the corresponding string "Invalid Date". If we go in this direction, conceivably we could even have Date.now() return NaN. The advantage of removing Date.now instead is to support the feature-testing style practiced by ECMAScript programmers.

  • Of course, there is the perpetual bikeshedding of names. We are not attached to the names we present here.

Spec Text

Updating the spec text for this proposal

The source for the spec text is located in spec/index.emu and it is written in ecmarkup language.

When modifying the spec text, you should be able to build the HTML version in index.html by using the following command:

npm install
npm run build
open index.html

Alternative, you can use npm run watch.

Acknowledgements

The Compartment API proposed here derives directly from Moddable's earlier Compartment API, in the XS implementation of standalone SES. We thank in particular Patrick Soquet and Peter Hoddlie for repeated sessions of brainstorming and refinement.

Thanks to the regular attendees at the recent SES meetings, especially Bradley Farias, Michael Fig, Saleh Motaal, and Chip Morningstar.

Many thanks to E. Dean Tribble, Kevin Reid, Dave Herman, Michael Ficarra, Tom Van Cutsem, Kris Kowal, Kevin Smith, Terry Hayes, Daniel Ehrenberg, Ojan Vafai, Elliott Sprehn, and Alex Russell. Thanks to the entire Caja team (Jasvir Nagra, Ihab Awad, Mike Stay, Mike Samuel, Felix Lee, Kevin Reid, and Ben Laurie) for building a system in which all the hardest issues have already been worked out.