Posts Tagged ‘nimbah’

Use of Knockout in Nimbah

December 27, 2012 Leave a comment

As well as Nimbah being a tool that does something I occasionally need, it’s also a chance to experiment. (Also by writing this up, I’m probably going to spot some things I can simplify.)

While I’ve been using Knockout¬†regularly in my work for a few months, it’s been as a tool that I apply here and there where appropriate. But in Nimbah I decided to go for broke and base the entire thing on Knockout. So in essence its a single view model, the whole document is bound once to the the view model, and when you edit the pipeline you are just editing the view model.

A simplified schematic version of the view model is:

var viewModel = {
    inputText: ko.observable(localStorage.getItem('savedInputText') || ''),
    selected: ko.observable(),
    pipeline: ko.observable(null),
    layout: ko.observable('vertical')

ko.computed(function() {
    var root = viewModel.pipeline();
    if (root) {

viewModel.outputText = ko.computed(function() {
    var root = viewModel.pipeline();
    return root ? viewModel.stringify(root.outputValue()) : '';

In the UI the inputText and outputText properties are each bound to a textarea (the output one being readonly).

But internally, thanks to the ko.computed parts, they are really little more than aliases for two properties on the root pipeline: inputValue and outputValue. The underlying model of Nimbah is based on nodes that all follow this same pattern. A pipeline is a node and operators are nodes, and some operators contain their own pipelines as child nodes, and so on.

The building block is the node function, which builds a raw node – again in simplified schematic form:

var node = function() {
    var model = {};
    model.readOnlyChildren = ko.observableArray();

    model.insertAfter = function(newChild, existingChild) ...
    model.insertBefore = function(newChild, existingChild) ...
    model.remove = function() ...
    model.parent = ko.computed({
        read: function() ...
        write: function(val) ...
    model.firstChild = ...
    model.lastChild = ...
    model.nextSibling = ...
    model.previousSibling = ...

    return model;

Each node has (possibly null) references to its parent, firstChild, lastChild, nextSibling and previousSibling, all of which are shielded by ko.computed with read/write operations so that they remain consistent. For example, if you assign node A to be the nextSibling of node B, that’s equivalent to B.parent().insertAfter(B, A), which ensures that B also becomes the previousSibling of A and that B’s parent is now also A’s parent.

There’s an observable array of the node’s children, called readOnlyChildren to try to emphasise that it isn’t meant to be directly modified. I have to expose it because it’s what the UI binds to, but its contents are automatically maintained (via individual insertions and deletions) by the above “proper” node properties.

Why do it this way? Because of the way nodes obtain their input values. If an operator has a previousSibling, it uses that sibling’s outputValue as its own inputValue. If it has a null previousSibling, it’s the first operator in its pipeline, so it should be using the pipeline’s inputValue. And guess what? The pipeline is its parent, so it has no problem getting the inputValue from it. Hence for any operator node, inputValue looks like this:

model.inputValue = ko.computed(function() {
    return model.previousSibling() ? model.previousSibling().outputValue() :
           model.parent() ? model.parent().inputValue() : null;

In a pipeline it’s a lot simpler:

model.inputValue = ko.observable(null);

This is because pipelines can act as the root of everything, or be used in whatever way an operator wants to use them. So it’s up to the owner of the pipeline to feed it with the right inputValue.

Of course, the outputValue of a node has to be entirely its own responsibility – the whole point of a node is how it turns a given input into the right output. For a pipeline, it’s just the last child node’s outputValue (or for an empty pipeline it’s just the inputValue):

model.outputValue = ko.computed(function() {
    return model.lastChild() ? model.lastChild().outputValue() : 

Each kind of operator has to implement outputValue differently. Here’s split:

model.outputValue = ko.computed(function() {
    var input = model.inputValue();
    if (input && typeof input.split == 'function') {
        return input.split(model.separator());
    return input;

So the upshot of all this is that whenever you edit the text in the input textarea, it ripples through all the operators in the pipeline completely automatically, through the miracle of ko.computed. If a node is unplugged from some location and then plugged in somewhere else, everything updates accordingly. It’s just beautiful!

And that’s before we even get on to the joy of how undo/redo just bolts onto the side without any real effort…

Categories: Uncategorized Tags: ,

A few extra operators in Nimbah

December 27, 2012 Leave a comment

I added skip and take operators that work exactly like their namesakes in Linq.¬†Also there’s a new substring operator that has a nice instant feedback UI:


Categories: Uncategorized Tags:

Nimbah in one minute

December 27, 2012 1 comment

Over the “holidays” I’ve been tinkering with a simple online tool that gives me a fast and easy way to reformat text (prompted by the fact that the macro recorder has been removed from Visual Studio 2012).

Here’s a very quick intro to it (will be clearer if you open it in YouTube fullscreen at 1080p).

You can play with it yourself here:

Categories: Uncategorized Tags:

Virtualized scrolling in knockout.js

December 26, 2012 Leave a comment

If you have an observableArray of things and you want to display it, of course the first thing to go for is the foreach binding. But what if the array is sometimes pretty long? It can take a long time for knockout to build the elements for each item. Once it’s in the thousands, it can freeze the UI for a while.

The classic-and-classy solution is to only ever create the handful of elements that are currently visible within the scrollable area. For example, here’s a plain old foreach binding to an array called all:

<div class="dataDisplay" data-bind="foreach: all">
    <div class="dataField" data-bind="text: name, drag: pick"></div>
    <div class="dataValue"><pre data-bind="text: value"></pre></div>

The first change I had to make was to put the list into a nested div. Why? Because I want to keep restricting the size of the dataDisplay using absolute positioning, but I’m also going to need to programmatically set the height of the scrollable content (which, as you’d expect, is going to be much bigger than the fixed height of the outer div):

<div class="dataDisplay">
    <div class="dataDisplayContent" data-bind="virtualScroll: { rows: all, rowHeight: 26 }">
        <div class="dataField" data-bind="text: name, drag: pick"></div>
        <div class="dataValue"><pre data-bind="text: value"></pre></div>

You may have noticed another change – the binding is now something called virtualScroll, and it takes an object with two properties: rows is just the same observableArray as before, and rowHeight is the pixel height of each row. This is the fiddly part of virtual scrolling: by far the easiest technique is to hold the height of the rows constant, as we’ll see.

So, how does the virtualScroll binding work? Like many effective bindings, it’s like a swan; on the surface the motion is graceful, but below the water things don’t look so tidy. In fact it’s a mess so I’ll tackle it one piece at a time. It starts in the usual way:

ko.bindingHandlers.virtualScroll = {
    init: function(element, valueAccessor, allBindingsAccessor, 
                   viewModel, context) {

The very first thing we have to do steal the contents of the target element. This will serve as our template for each item in the list, so we clone our own copy so we can use it later:

        var clone = $(element).clone();

Then we grab our settings. I name the whole object config, and I immediately unwrap the rowHeight so I can conveniently use it in several places:

        var config = ko.utils.unwrapObservable(valueAccessor());
        var rowHeight = ko.utils.unwrapObservable(config.rowHeight);

Now, we get the length of the rows array, multiply by the fixed rowHeight, and so set the height of our target element, which typically makes it bigger than the container div, which should then get scrollbars (because it should have overflow: auto set in CSS).

But thanks to the magic of ko.computed, knockout notices that we access the rows() observable and so arranges for the function to run again when the observable’s value changes. And so when the length of the array changes, so does the height of our scrollable content. (This would be a little more effort if the rows could vary in height).

        ko.computed(function() {
                height: config.rows().length * rowHeight

Now, ko.computed is wonderful, but it can only work with existing observables. Sadly, the browser DOM is full of useful information that doesn’t tell us when it changes. As a hack-around, I use a little helper function called simulatedObservable, which you can read about here. For now, just accept that offset and windowHeight are observables that track the y-offset of the target element, and the height of the whole window.

        var offset = simulatedObservable(element, function() {
            return $(element).offset().top;

        var windowHeight = simulatedObservable(element, function() {
            return window.innerHeight;

We have all the external info we need. Next we have to maintain our own state, which is a record of all the rows that we have “materialised” into real DOM elements. The others simply don’t exist.

        var created = {};

And now we begin a rather large nested function called refresh, responsible for materialising any rows that are currently visible.

        var refresh = function() {
            var o = offset();
            var data = config.rows();
            var top = Math.max(0, Math.floor(-o / rowHeight) - 10);
            var bottom = Math.min(data.length, Math.ceil((-o + windowHeight()) / rowHeight));

The top and bottom variables are the indexes of the first and one-after-last rows that we consider visible (actually we’re being quite generous because we use the height of the whole window as the bottom bound). This would be a lot more troublesome if the rows could vary in height.

So we can loop through just these rows, clone our “template” and ask knockout to bind to it, using the row’s item from the array as the data model.

            for (var row = top; row < bottom; row++) {
                if (!created[row]) {
                    var rowDiv = $('<div></div>');
                        position: 'absolute',
                        height: config.rowHeight,
                        left: 0,
                        right: 0,
                        top: row * config.rowHeight
                        context.createChildContext(data[row]), rowDiv[0]);
                    created[row] = rowDiv;

Finally, we can destroy any previously materialised row that is not in the currently visible range:

            Object.keys(created).forEach(function(rowNum) {
                if (rowNum < top || rowNum >= bottom) {
                    delete created[rowNum];

And that concludes the refresh function. There are two places we call it. First, when the rows() observable changes, we clear out all materialised rows, and then call refresh.

        config.rows.subscribe(function() {
            Object.keys(created).forEach(function(rowNum) {
                delete created[rowNum];

The second place is a little simpler: we just give it to ko.computed so it can do its usual magic and ensure that refresh runs whenever there is a change in any of the values it depends on.


Finally, we tell Knockout not to automatically perform binding on the child elements, as we take care of that ourselves:

        return { controlsDescendantBindings: true };

You can see this in action in my Nimbah project (source code here), where I use it to display the input or output data for the selected node, allowing it to scale to thousands of rows without slowing down the UI.