From 30a5cfc6c8dfe0ff412e63498eac09d31e9004a7 Mon Sep 17 00:00:00 2001 From: Klaus Weidner Date: Thu, 6 Jun 2013 16:38:12 -0400 Subject: [PATCH] Refactoring to fix stacked graphs with NaNs. For stacked graphs, draw gaps for just the specific parts of series that are missing data, and use interpolation to help ensure that the overall graph shape doesn't get distorted for missing points. This replaces #139 "Fix NaN handling for stacked graphs" which had aggressively propagated NaNs to ensure that graphs don't get drawn with incorrect stacking. Fixes issue 446 - Improve handling of NaNs in stacked graphs. Detailed changes: * Remove layouts_.datasets, instead create point object arrays directly from gatherDatasets_(). This was necessary since the pre-stacked datasets aren't able to store the additional data needed for handling gaps. * For stacked graphs, track yval_stacked/y_stacked separately from yval as point properties. * Remove unstackPointAtIndex which is now no longer necessary, since the points keep the original yval. This helps ensure that the returned values keep their original NaN value and don't expose the interpolated stacked values used for drawing gaps. * Remove evaluateWithError, evaluateLineCharts and Dygraph.seriesToPoints_ now handle error bars directly. * In gatherDatasets_, remove the unconditional copy of rolledSeries, the refactoring appears to have fixed the bug related to zooming with error bars on. (I could reproduce it in the original code by zooming dygraphs/tests/custom-bars.html.) For date windows (horizontal zoom), use .splice() instead of manually coping. Squashed commit of the following: commit 56efaef6a50c737b99a8e4e6a36e55c83fde237f Author: Klaus Weidner Date: Mon Jun 3 16:50:16 2013 -0700 Style fixes as per Robert's requests. commit 83bb38a071f407560abcca53b7f6e07d6b0ca0e5 Author: Klaus Weidner Date: Mon Jun 3 15:53:59 2013 -0700 Add stackedGraphNaNFill option to control NaN handling. Add a test for this, and add a docstring to the stackPoints_ method. commit e16e3c381d5cc7bec9896b31dddebc627a941c33 Author: Klaus Weidner Date: Mon Jun 3 15:20:24 2013 -0700 Revert name back to .addDataset, add docstring. commit d3e66a60c84656364829e8a5bb83ed0e2b9fb403 Author: Klaus Weidner Date: Mon Jun 3 15:15:19 2013 -0700 Add more type annotations to make the flow of point data clearer. Unfortunately it's still not very clear, the logic is a bit convoluted. commit 33db3d27501c8b5f2f84baa4dd92bf8d6f0a35a3 Author: Klaus Weidner Date: Mon Jun 3 15:00:38 2013 -0700 Rename fixPathAttrs_ to cleanPathAttrs_ as requested. commit f39168785efcbacd1de9a6275aa02c8e34e4c5a7 Author: Klaus Weidner Date: Mon Jun 3 14:58:26 2013 -0700 Add comment explaining baseline, as requested by Robert. commit 80348ed3c9e677fb5ff1ef393056e1e2bea1bbf7 Author: Klaus Weidner Date: Mon Jun 3 13:16:51 2013 -0700 Track stroke/fill in CanvasAssertions. For lineTo operations, set strokeStyle to undefined if the path is filled (not stroked), and set fillStyle to undefined if the path is stroked only. Fix assertions in missing_points tests, those were apparently expecting the inflated numbers. Remove the strokeStyle='#000000' hack in dygraph-canvas which is now no longer needed. commit 4f47af82ac297d27dc0ac9d5c6fd256e00cccec5 Author: Klaus Weidner Date: Mon Jun 3 11:57:56 2013 -0700 fix CanvasAssertions which was calling .match wrong, remove now-redundant check. commit 8c53e9e0f3d362e58204ff5a777c90ddb3b589d5 Author: Klaus Weidner Date: Thu May 23 11:00:50 2013 -0700 Restore hasOwnProperty. commit 621cc37cfaf33821d04de2a94aea63021f7a9647 Author: Klaus Weidner Date: Wed May 22 15:35:16 2013 -0700 Fix all-NaN series stacking and Proxy return values. Ensure that all-NaN series get treated as zero for stacking purposes, and that nextPoint gets set to null past the last stackable point. We want interpolation, but not extrapolation. The Proxy class didn't return values from calls, breaking testCorrectColors since it didn't get pixel data as expected. Update the input data for testInterpolation to check corner cases, including an all-NaN series. commit eb52ff1c1bbdd55312cb2881b5182731c91c155c Author: Klaus Weidner Date: Wed May 22 12:40:44 2013 -0700 Refactoring to fix stacked graphs with NaNs. For stacked graphs, draw gaps for just the specific parts of series that are missing data, and use interpolation to help ensure that the overall graph shape doesn't get distorted for missing points. This replaces https://github.com/danvk/dygraphs/pull/139 "Fix NaN handling for stacked graphs" which had aggressively propagated NaNs to ensure that graphs don't get drawn with incorrect stacking. Fixes issue 446 - Improve handling of NaNs in stacked graphs. Detailed changes: - Remove layouts_.datasets, instead create point object arrays directly from gatherDatasets_(). This was necessary since the pre-stacked datasets aren't able to store the additional data needed for handling gaps. - For stacked graphs, track yval_stacked/y_stacked separately from yval as point properties. - Remove unstackPointAtIndex which is now no longer necessary, since the points keep the original yval. This helps ensure that the returned values keep their original NaN value and don't expose the interpolated stacked values used for drawing gaps. - Remove evaluateWithError, evaluateLineCharts and Dygraph.seriesToPoints_ now handle error bars directly. - In gatherDatasets_, remove the unconditional copy of rolledSeries, the refactoring appears to have fixed the bug related to zooming with error bars on. (I could reproduce it in the original code by zooming dygraphs/tests/custom-bars.html.) For date windows (horizontal zoom), use .splice() instead of manually coping. --- auto_tests/tests/CanvasAssertions.js | 35 +++- auto_tests/tests/Proxy.js | 3 +- auto_tests/tests/callback.js | 16 +- auto_tests/tests/missing_points.js | 8 +- auto_tests/tests/stacked.js | 100 +++++++++++ dygraph-canvas.js | 12 +- dygraph-gviz.js | 4 +- dygraph-layout.js | 162 ++++++------------ dygraph-options-reference.js | 8 +- dygraph.js | 313 +++++++++++++++++++++++------------ 10 files changed, 422 insertions(+), 239 deletions(-) diff --git a/auto_tests/tests/CanvasAssertions.js b/auto_tests/tests/CanvasAssertions.js index e8bb3e3..929ba4f 100644 --- a/auto_tests/tests/CanvasAssertions.js +++ b/auto_tests/tests/CanvasAssertions.js @@ -27,6 +27,36 @@ var CanvasAssertions = {}; /** + * Updates path attributes to match fill/stroke operations. + * + * This sets fillStyle to undefined for stroked paths, + * and strokeStyle to undefined for filled paths, to simplify + * matchers such as numLinesDrawn. + * + * @private + * @param {Array.} List of operations. + */ +CanvasAssertions.cleanPathAttrs_ = function(calls) { + var isStroked = true; + for (var i = calls.length - 1; i >= 0; --i) { + var call = calls[i]; + var name = call.name; + if (name == 'stroke') { + isStroked = true; + } else if (name == 'fill') { + isStroked = false; + } else if (name == 'lineTo') { + if (isStroked) { + call.properties.fillStyle = undefined; + } else { + call.properties.strokeStyle = undefined; + } + } + } +}; + + +/** * Assert that a line is drawn between the two points * * This merely looks for one of these four possibilities: @@ -40,6 +70,7 @@ var CanvasAssertions = {}; * or a function that accepts the current call. */ CanvasAssertions.assertLineDrawn = function(proxy, p1, p2, predicate) { + CanvasAssertions.cleanPathAttrs_(proxy.calls__); // found = 1 when prior loop found p1. // found = 2 when prior loop found p2. var priorFound = 0; @@ -62,7 +93,7 @@ CanvasAssertions.assertLineDrawn = function(proxy, p1, p2, predicate) { } } if (priorFound == 2 && matchp1) { - if (CanvasAssertions.match(predicate, call.properties)) { + if (CanvasAssertions.match(predicate, call)) { return; } } @@ -101,6 +132,7 @@ CanvasAssertions.assertLineDrawn = function(proxy, p1, p2, predicate) { * color and stroke width. */ CanvasAssertions.getLinesDrawn = function(proxy, predicate) { + CanvasAssertions.cleanPathAttrs_(proxy.calls__); var lastCall; var lines = []; for (var i = 0; i < proxy.calls__.length; i++) { @@ -149,6 +181,7 @@ CanvasAssertions.assertBalancedSaveRestore = function(proxy) { // common case. Possibly allow predicate to be function, hash, or // string representing color? CanvasAssertions.numLinesDrawn = function(proxy, color) { + CanvasAssertions.cleanPathAttrs_(proxy.calls__); var num_lines = 0; for (var i = 0; i < proxy.calls__.length; i++) { var call = proxy.calls__[i]; diff --git a/auto_tests/tests/Proxy.js b/auto_tests/tests/Proxy.js index 5c03dc6..852009e 100644 --- a/auto_tests/tests/Proxy.js +++ b/auto_tests/tests/Proxy.js @@ -38,7 +38,7 @@ var Proxy = function(delegate) { function makeFunc(name) { return function() { this.log__(name, arguments); - this.delegate__[name].apply(this.delegate__, arguments); + return this.delegate__[name].apply(this.delegate__, arguments); } }; this[propname] = makeFunc(propname); @@ -71,4 +71,3 @@ Proxy.prototype.log__ = function(name, args) { var call = { name : name, args : args, properties: properties }; this.calls__.push(call); }; - diff --git a/auto_tests/tests/callback.js b/auto_tests/tests/callback.js index a8d5e7e..af524a0 100644 --- a/auto_tests/tests/callback.js +++ b/auto_tests/tests/callback.js @@ -420,17 +420,23 @@ CallbackTestCase.prototype.testNaNDataStack = function() { assertEquals(1, res.row); assertEquals('c', res.seriesName); - // First gap, no data due to NaN contagion. + // All-NaN area at left, should get no points. + dom = g.toDomCoords(9.1, 0.9); + res = g.findStackedPoint(dom[0], dom[1]); + assertEquals(0, res.row); + assertEquals(undefined, res.seriesName); + + // First gap, get 'c' since it's non-NaN. dom = g.toDomCoords(12.1, 0.9); res = g.findStackedPoint(dom[0], dom[1]); assertEquals(3, res.row); - assertEquals(undefined, res.seriesName); + assertEquals('c', res.seriesName); - // Second gap, no data due to NaN contagion. + // Second gap, get 'b' since 'c' is NaN. dom = g.toDomCoords(15.1, 0.9); res = g.findStackedPoint(dom[0], dom[1]); assertEquals(6, res.row); - assertEquals(undefined, res.seriesName); + assertEquals('b', res.seriesName); // Isolated points should work, finding series b in this case. dom = g.toDomCoords(15.9, 3.1); @@ -685,4 +691,4 @@ CallbackTestCase.prototype.testDrawHighlightPointCallback_idx = function() { assertEquals(0,idxToCheck); DygraphOps.dispatchMouseMove(g, 6, 3); assertEquals(5,idxToCheck); -}; \ No newline at end of file +}; diff --git a/auto_tests/tests/missing_points.js b/auto_tests/tests/missing_points.js index 3c52dd0..cf468f9 100644 --- a/auto_tests/tests/missing_points.js +++ b/auto_tests/tests/missing_points.js @@ -188,7 +188,7 @@ MissingPointsTestCase.prototype.testErrorBarsWithMissingPoints = function() { var htx = g.hidden_ctx_; - assertEquals(8, CanvasAssertions.numLinesDrawn(htx, '#ff0000')); + assertEquals(2, CanvasAssertions.numLinesDrawn(htx, '#ff0000')); var p0 = g.toDomCoords(data[0][0], data[0][1][0]); var p1 = g.toDomCoords(data[1][0], data[1][1][0]); @@ -222,7 +222,7 @@ MissingPointsTestCase.prototype.testErrorBarsWithMissingPointsConnected = functi var htx = g.hidden_ctx_; - assertEquals(8, CanvasAssertions.numLinesDrawn(htx, '#ff0000')); + assertEquals(2, CanvasAssertions.numLinesDrawn(htx, '#ff0000')); var p1 = g.toDomCoords(data[1][0], data[1][1][0]); var p2 = g.toDomCoords(data[3][0], data[3][1][0]); @@ -257,7 +257,7 @@ MissingPointsTestCase.prototype.testCustomBarsWithMissingPoints = function() { var htx = g.hidden_ctx_; - assertEquals(16, CanvasAssertions.numLinesDrawn(htx, '#ff0000')); + assertEquals(4, CanvasAssertions.numLinesDrawn(htx, '#ff0000')); var p0 = g.toDomCoords(data[0][0], data[0][1][1]); var p1 = g.toDomCoords(data[1][0], data[1][1][1]); @@ -298,7 +298,7 @@ MissingPointsTestCase.prototype.testCustomBarsWithMissingPointsConnected = funct var htx = g.hidden_ctx_; - assertEquals(8, CanvasAssertions.numLinesDrawn(htx, '#ff0000')); + assertEquals(2, CanvasAssertions.numLinesDrawn(htx, '#ff0000')); var p1 = g.toDomCoords(data[1][0], data[1][1][1]); var p2 = g.toDomCoords(data[3][0], data[3][1][1]); diff --git a/auto_tests/tests/stacked.js b/auto_tests/tests/stacked.js index e365c58..b8c095e 100644 --- a/auto_tests/tests/stacked.js +++ b/auto_tests/tests/stacked.js @@ -5,11 +5,17 @@ */ var stackedTestCase = TestCase("stacked"); +stackedTestCase._origGetContext = Dygraph.getContext; + stackedTestCase.prototype.setUp = function() { document.body.innerHTML = "
"; + Dygraph.getContext = function(canvas) { + return new Proxy(stackedTestCase._origGetContext(canvas)); + } }; stackedTestCase.prototype.tearDown = function() { + Dygraph.getContext = stackedTestCase._origGetContext; }; stackedTestCase.prototype.testCorrectColors = function() { @@ -179,3 +185,97 @@ stackedTestCase.prototype.testMissingValueAtZero = function() { g.setSelection(2); assertEquals("2: Y2: 3", Util.getLegend()); }; + +stackedTestCase.prototype.testInterpolation = function() { + var opts = { + colors: ['#ff0000', '#00ff00', '#0000ff'], + stackedGraph: true + }; + + // The last series is all-NaN, it ought to be treated as all zero + // for stacking purposes. + var N = NaN; + var data = [ + [100, 1, 2, N, N], + [101, 1, 2, 2, N], + [102, 1, N, N, N], + [103, 1, 2, 4, N], + [104, N, N, N, N], + [105, 1, 2, N, N], + [106, 1, 2, 7, N], + [107, 1, 2, 8, N], + [108, 1, 2, 9, N], + [109, 1, N, N, N]]; + + var graph = document.getElementById("graph"); + g = new Dygraph(graph, data, opts); + + var htx = g.hidden_ctx_; + var attrs = {}; + + // Check that lines are drawn at the expected positions, using + // interpolated values for missing data. + CanvasAssertions.assertLineDrawn( + htx, g.toDomCoords(100, 4), g.toDomCoords(101, 4), {strokeStyle: '#00ff00'}); + CanvasAssertions.assertLineDrawn( + htx, g.toDomCoords(102, 6), g.toDomCoords(103, 7), {strokeStyle: '#ff0000'}); + CanvasAssertions.assertLineDrawn( + htx, g.toDomCoords(107, 8), g.toDomCoords(108, 9), {strokeStyle: '#0000ff'}); + CanvasAssertions.assertLineDrawn( + htx, g.toDomCoords(108, 12), g.toDomCoords(109, 12), {strokeStyle: '#ff0000'}); + + // Check that the expected number of line segments gets drawn + // for each series. Gaps don't get a line. + assertEquals(7, CanvasAssertions.numLinesDrawn(htx, '#ff0000')); + assertEquals(4, CanvasAssertions.numLinesDrawn(htx, '#00ff00')); + assertEquals(2, CanvasAssertions.numLinesDrawn(htx, '#0000ff')); + + // Check that the selection returns the original (non-stacked) + // values and skips gaps. + g.setSelection(1); + assertEquals("101: Y1: 1 Y2: 2 Y3: 2", Util.getLegend()); + + g.setSelection(8); + assertEquals("108: Y1: 1 Y2: 2 Y3: 9", Util.getLegend()); + + g.setSelection(9); + assertEquals("109: Y1: 1", Util.getLegend()); +}; + +stackedTestCase.prototype.testInterpolationOptions = function() { + var opts = { + colors: ['#ff0000', '#00ff00', '#0000ff'], + stackedGraph: true + }; + + var data = [ + [100, 1, NaN, 3], + [101, 1, 2, 3], + [102, 1, NaN, 3], + [103, 1, 2, 3], + [104, 1, NaN, 3]]; + + var choices = ['all', 'inside', 'none']; + var stackedY = [ + [6, 6, 6, 6, 6], + [4, 6, 6, 6, 4], + [4, 6, 4, 6, 4]]; + + for (var i = 0; i < choices.length; ++i) { + var graph = document.getElementById("graph"); + opts['stackedGraphNaNFill'] = choices[i]; + g = new Dygraph(graph, data, opts); + + var htx = g.hidden_ctx_; + var attrs = {}; + + // Check top lines get drawn at the expected positions. + for (var j = 0; j < stackedY[i].length - 1; ++j) { + CanvasAssertions.assertLineDrawn( + htx, + g.toDomCoords(100 + j, stackedY[i][j]), + g.toDomCoords(101 + j, stackedY[i][j + 1]), + {strokeStyle: '#ff0000'}); + } + } +}; diff --git a/dygraph-canvas.js b/dygraph-canvas.js index ef6cf1f..cba4f44 100644 --- a/dygraph-canvas.js +++ b/dygraph-canvas.js @@ -702,7 +702,14 @@ DygraphCanvasRenderer._fillPlotter = function(e) { var stackedGraph = g.getOption("stackedGraph"); var colors = g.getColors(); - var baseline = {}; // for stacked graphs: baseline for filling + // For stacked graphs, track the baseline for filling. + // + // The filled areas below graph lines are trapezoids with two + // vertical edges. The top edge is the line segment being drawn, and + // the baseline is the bottom edge. Each baseline corresponds to the + // top line segment from the previous stacked line. In the case of + // step plots, the trapezoids are rectangles. + var baseline = {}; var currBaseline; var prevStepPlot; // for different line drawing modes (line/step) per series @@ -739,6 +746,9 @@ DygraphCanvasRenderer._fillPlotter = function(e) { var point = iter.next(); if (!Dygraph.isOK(point.y)) { prevX = NaN; + if (point.y_stacked !== null && !isNaN(point.y_stacked)) { + baseline[point.canvasx] = area.h * point.y_stacked + area.y; + } continue; } if (stackedGraph) { diff --git a/dygraph-gviz.js b/dygraph-gviz.js index 49feac5..988e0ac 100644 --- a/dygraph-gviz.js +++ b/dygraph-gviz.js @@ -73,8 +73,8 @@ Dygraph.GVizChart.prototype.getSelection = function() { if (row < 0) return selection; - var datasets = this.date_graph.layout_.datasets; - for (var setIdx = 0; setIdx < datasets.length; ++setIdx) { + var points = this.date_graph.layout_.points; + for (var setIdx = 0; setIdx < points.length; ++setIdx) { selection.push({row: row, column: setIdx + 1}); } diff --git a/dygraph-layout.js b/dygraph-layout.js index 54496aa..cf94818 100644 --- a/dygraph-layout.js +++ b/dygraph-layout.js @@ -31,11 +31,21 @@ */ var DygraphLayout = function(dygraph) { this.dygraph_ = dygraph; - this.datasets = []; + /** + * Array of points for each series. + * + * [series index][row index in series] = |Point| structure, + * where series index refers to visible series only, and the + * point index is for the reduced set of points for the current + * zoom region (including one point just outside the window). + * All points in the same row index share the same X value. + * + * @type {Array.>} + */ + this.points = []; this.setNames = []; this.annotations = []; this.yAxes_ = null; - this.points = null; // TODO(danvk): it's odd that xTicks_ and yTicks_ are inputs, but xticks and // yticks are outputs. Clean this up. @@ -47,8 +57,14 @@ DygraphLayout.prototype.attr_ = function(name) { return this.dygraph_.attr_(name); }; +/** + * Add points for a single series. + * + * @param {string} setname Name of the series. + * @param {Array.} set_xy Points for the series. + */ DygraphLayout.prototype.addDataset = function(setname, set_xy) { - this.datasets.push(set_xy); + this.points.push(set_xy); this.setNames.push(setname); }; @@ -205,52 +221,46 @@ DygraphLayout._calcYNormal = function(axis, value, logscale) { DygraphLayout.prototype._evaluateLineCharts = function() { var connectSeparated = this.attr_('connectSeparatedPoints'); + var isStacked = this.attr_("stackedGraph"); + var hasBars = this.attr_('errorBars') || this.attr_('customBars'); - // series index -> point index in series -> |point| structure - this.points = new Array(this.datasets.length); - - // TODO(bhs): these loops are a hot-spot for high-point-count charts. In fact, - // on chrome+linux, they are 6 times more expensive than iterating through the - // points and drawing the lines. The brunt of the cost comes from allocating - // the |point| structures. var boundaryIdStart = this.dygraph_.getLeftBoundary_(); - for (var setIdx = 0; setIdx < this.datasets.length; setIdx++) { - var dataset = this.datasets[setIdx]; + for (var setIdx = 0; setIdx < this.points.length; setIdx++) { + var points = this.points[setIdx]; var setName = this.setNames[setIdx]; var axis = this.dygraph_.axisPropertiesForSeries(setName); // TODO (konigsberg): use optionsForAxis instead. var logscale = this.dygraph_.attributes_.getForSeries("logscale", setName); - // Preallocating the size of points reduces reallocations, and therefore, - // calls to collect garbage. - var seriesPoints = new Array(dataset.length); - - for (var j = 0; j < dataset.length; j++) { - var item = dataset[j]; - var xValue = DygraphLayout.parseFloat_(item[0]); - var yValue = DygraphLayout.parseFloat_(item[1]); + for (var j = 0; j < points.length; j++) { + var point = points[j]; // Range from 0-1 where 0 represents left and 1 represents right. - var xNormal = (xValue - this.minxval) * this.xscale; + point.x = (point.xval - this.minxval) * this.xscale; // Range from 0-1 where 0 represents top and 1 represents bottom - var yNormal = DygraphLayout._calcYNormal(axis, yValue, logscale); + var yval = point.yval; + if (isStacked) { + point.y_stacked = DygraphLayout._calcYNormal( + axis, point.yval_stacked, logscale); + if (yval !== null && !isNaN(yval)) { + yval = point.yval_stacked; + } + } + if (yval === null) { + yval = NaN; + if (!connectSeparated) { + point.yval = NaN; + } + } + point.y = DygraphLayout._calcYNormal(axis, yval, logscale); - // TODO(danvk): drop the point in this case, don't null it. - // The nulls create complexity in DygraphCanvasRenderer._drawSeries. - if (connectSeparated && item[1] === null) { - yValue = null; + if (hasBars) { + point.y_top = DygraphLayout._calcYNormal( + axis, yval - point.yval_minus, logscale); + point.y_bottom = DygraphLayout._calcYNormal( + axis, yval + point.yval_plus, logscale); } - seriesPoints[j] = { - x: xNormal, - y: yNormal, - xval: xValue, - yval: yValue, - name: setName, // TODO(danvk): is this really necessary? - idx: j + boundaryIdStart - }; } - - this.points[setIdx] = seriesPoints; } }; @@ -294,45 +304,6 @@ DygraphLayout.prototype._evaluateLineTicks = function() { } }; - -/** - * Behaves the same way as PlotKit.Layout, but also copies the errors - * @private - */ -DygraphLayout.prototype.evaluateWithError = function() { - this.evaluate(); - if (!(this.attr_('errorBars') || this.attr_('customBars'))) return; - - // Copy over the error terms - var i = 0; // index in this.points - for (var setIdx = 0; setIdx < this.datasets.length; ++setIdx) { - var points = this.points[setIdx]; - var j = 0; - var dataset = this.datasets[setIdx]; - var setName = this.setNames[setIdx]; - var axis = this.dygraph_.axisPropertiesForSeries(setName); - // TODO (konigsberg): use optionsForAxis instead. - var logscale = this.dygraph_.attributes_.getForSeries("logscale", setName); - - for (j = 0; j < dataset.length; j++, i++) { - var item = dataset[j]; - var xv = DygraphLayout.parseFloat_(item[0]); - var yv = DygraphLayout.parseFloat_(item[1]); - - if (xv == points[j].xval && - yv == points[j].yval) { - var errorMinus = DygraphLayout.parseFloat_(item[2]); - var errorPlus = DygraphLayout.parseFloat_(item[3]); - - var yv_minus = yv - errorMinus; - var yv_plus = yv + errorPlus; - points[j].y_top = DygraphLayout._calcYNormal(axis, yv_minus, logscale); - points[j].y_bottom = DygraphLayout._calcYNormal(axis, yv_plus, logscale); - } - } - } -}; - DygraphLayout.prototype._evaluateAnnotations = function() { // Add the annotations to the point to which they belong. // Make a map from (setName, xval) to annotation for quick lookups. @@ -368,49 +339,12 @@ DygraphLayout.prototype._evaluateAnnotations = function() { * Convenience function to remove all the data sets from a graph */ DygraphLayout.prototype.removeAllDatasets = function() { - delete this.datasets; + delete this.points; delete this.setNames; delete this.setPointsLengths; delete this.setPointsOffsets; - this.datasets = []; + this.points = []; this.setNames = []; this.setPointsLengths = []; this.setPointsOffsets = []; }; - -/** - * Return a copy of the point at the indicated index, with its yval unstacked. - * @param int index of point in layout_.points - */ -DygraphLayout.prototype.unstackPointAtIndex = function(setIdx, row) { - var point = this.points[setIdx][row]; - // If the point is missing, no unstacking is necessary - if (!Dygraph.isValidPoint(point)) { - return point; - } - - // Clone the point since we modify it - var unstackedPoint = {}; - for (var pt in point) { - unstackedPoint[pt] = point[pt]; - } - - if (!this.attr_("stackedGraph")) { - return unstackedPoint; - } - - // The unstacked yval is equal to the current yval minus the yval of the - // next point at the same xval. - // We need to iterate over setIdx just in case some series have invalid values - // at current row - for(setIdx++; setIdx < this.points.length; setIdx++) { - var nextPoint = this.points[setIdx][row]; - if (nextPoint.xval == point.xval && // should always be true? - Dygraph.isValidPoint(nextPoint)) { - unstackedPoint.yval -= nextPoint.yval; - break; // stop at first valid point - } - } - - return unstackedPoint; -}; diff --git a/dygraph-options-reference.js b/dygraph-options-reference.js index bd69bfc..64e738d 100644 --- a/dygraph-options-reference.js +++ b/dygraph-options-reference.js @@ -23,7 +23,13 @@ Dygraph.OPTIONS_REFERENCE = // "default": "false", "labels": ["Data Line display"], "type": "boolean", - "description": "If set, stack series on top of one another rather than drawing them independently. The first series specified in the input data will wind up on top of the chart and the last will be on bottom." + "description": "If set, stack series on top of one another rather than drawing them independently. The first series specified in the input data will wind up on top of the chart and the last will be on bottom. NaN values are drawn as white areas without a line on top, see stackedGraphNaNFill for details." + }, + "stackedGraphNaNFill": { + "default": "all", + "labels": ["Data Line display"], + "type": "string", + "description": "Controls handling of NaN values inside a stacked graph. NaN values are interpolated/extended for stacking purposes, but the actual point value remains NaN in the legend display. Valid option values are \"all\" (interpolate internally, repeat leftmost and rightmost value as needed), \"inside\" (interpolate internally only, use zero outside leftmost and rightmost value), and \"none\" (treat NaN as zero everywhere)." }, "pointSize": { "default": "1", diff --git a/dygraph.js b/dygraph.js index dc3b798..ffffa01 100644 --- a/dygraph.js +++ b/dygraph.js @@ -291,6 +291,7 @@ Dygraph.DEFAULT_ATTRS = { connectSeparatedPoints: false, stackedGraph: false, + stackedGraphNaNFill: 'all', hideOverlayOnMouseOut: true, // TODO(danvk): support 'onmouseover' and 'never', and remove synonyms. @@ -1578,7 +1579,7 @@ Dygraph.prototype.resetZoom = function() { oldValueRanges = this.yAxisRanges(); // TODO(danvk): this is pretty inefficient var packed = this.gatherDatasets_(this.rolledSeries_, null); - var extremes = packed[1]; + var extremes = packed.extremes; // this has the side-effect of modifying this.axes_. // this doesn't make much sense in this context, but it's convenient (we @@ -1727,7 +1728,7 @@ Dygraph.prototype.findClosestPoint = function(domX, domY) { var minDist = Infinity; var idx = -1; var dist, dx, dy, point, closestPoint, closestSeries; - for ( var setIdx = this.layout_.datasets.length - 1 ; setIdx >= 0 ; --setIdx ) { + for ( var setIdx = this.layout_.points.length - 1 ; setIdx >= 0 ; --setIdx ) { var points = this.layout_.points[setIdx]; for (var i = 0; i < points.length; ++i) { var point = points[i]; @@ -1768,7 +1769,7 @@ Dygraph.prototype.findStackedPoint = function(domX, domY) { var boundary = this.getLeftBoundary_(); var rowIdx = row - boundary; var closestPoint, closestSeries; - for (var setIdx = 0; setIdx < this.layout_.datasets.length; ++setIdx) { + for (var setIdx = 0; setIdx < this.layout_.points.length; ++setIdx) { var points = this.layout_.points[setIdx]; if (rowIdx >= points.length) continue; var p1 = points[rowIdx]; @@ -1874,14 +1875,6 @@ Dygraph.prototype.idxToRow_ = function(setIdx, rowIdx) { var boundary = this.getLeftBoundary_(); return boundary + rowIdx; - // for (var setIdx = 0; setIdx < this.layout_.datasets.length; ++setIdx) { - // var set = this.layout_.datasets[setIdx]; - // if (idx < set.length) { - // return boundary + idx; - // } - // idx -= set.length; - // } - // return -1; }; Dygraph.prototype.animateSelection_ = function(direction) { @@ -2021,15 +2014,10 @@ Dygraph.prototype.setSelection = function(row, opt_seriesName, opt_locked) { if (row !== false && row >= 0) { if (row != this.lastRow_) changed = true; this.lastRow_ = row; - for (var setIdx = 0; setIdx < this.layout_.datasets.length; ++setIdx) { - var set = this.layout_.datasets[setIdx]; - if (row < set.length) { - var point = this.layout_.points[setIdx][row]; - - if (this.attr_("stackedGraph")) { - point = this.layout_.unstackPointAtIndex(setIdx, row); - } - + for (var setIdx = 0; setIdx < this.layout_.points.length; ++setIdx) { + var points = this.layout_.points[setIdx]; + if (row < points.length) { + var point = points[row]; if (point.yval !== null) this.selPoints_.push(point); } } @@ -2271,6 +2259,163 @@ Dygraph.prototype.predraw_ = function() { }; /** + * Point structure. + * + * xval_* and yval_* are the original unscaled data values, + * while x_* and y_* are scaled to the range (0.0-1.0) for plotting. + * yval_stacked is the cumulative Y value used for stacking graphs, + * and bottom/top/minus/plus are used for error bar graphs. + * + * @typedef {{ + * idx: number, + * name: string, + * x: ?number, + * xval: ?number, + * y_bottom: ?number, + * y: ?number, + * y_stacked: ?number, + * y_top: ?number, + * yval_minus: ?number, + * yval: ?number, + * yval_plus: ?number, + * yval_stacked + * }} + */ +Dygraph.PointType; + +// TODO(bhs): these loops are a hot-spot for high-point-count charts. In fact, +// on chrome+linux, they are 6 times more expensive than iterating through the +// points and drawing the lines. The brunt of the cost comes from allocating +// the |point| structures. +/** + * Converts a series to a Point array. + * + * @param {Array.)>} series Array where + * series[row] = [x,y] or [x, [y, err]] or [x, [y, yplus, yminus]]. + * @param {boolean} bars True if error bars or custom bars are being drawn. + * @param {string} setName Name of the series. + * @param {number} boundaryIdStart Index offset of the first point, equal to + * the number of skipped points left of the date window minimum (if any). + * @return {Array.} List of points for this series. + */ +Dygraph.seriesToPoints_ = function(series, bars, setName, boundaryIdStart) { + var points = []; + for (var i = 0; i < series.length; ++i) { + var item = series[i]; + var yraw = bars ? item[1][0] : item[1]; + var yval = yraw === null ? null : DygraphLayout.parseFloat_(yraw); + var point = { + x: NaN, + y: NaN, + xval: DygraphLayout.parseFloat_(item[0]), + yval: yval, + name: setName, // TODO(danvk): is this really necessary? + idx: i + boundaryIdStart + }; + + if (bars) { + point.y_top = NaN, + point.y_bottom = NaN, + point.yval_minus = DygraphLayout.parseFloat_(item[1][1]); + point.yval_plus = DygraphLayout.parseFloat_(item[1][2]); + } + points.push(point); + } + return points; +}; + + +/** + * Calculates point stacking for stackedGraph=true. + * + * For stacking purposes, interpolate or extend neighboring data across + * NaN values based on stackedGraphNaNFill settings. This is for display + * only, the underlying data value as shown in the legend remains NaN. + * + * @param {Array.} points Point array for a single series. + * Updates each Point's yval_stacked property. + * @param {Array.} cumulativeYval Accumulated top-of-graph stacked Y + * values for the series seen so far. Index is the row number. Updated + * based on the current series's values. + * @param {Array.} seriesExtremes Min and max values, updated + * to reflect the stacked values. + * @param {string} fillMethod Interpolation method, one of 'all', 'inside', or + * 'none'. + */ +Dygraph.stackPoints_ = function( + points, cumulativeYval, seriesExtremes, fillMethod) { + var lastXval = null; + var prevPoint = null; + var nextPoint = null; + var nextPointIdx = -1; + + // Find the next stackable point starting from the given index. + function updateNextPoint(idx) { + // If we've previously found a non-NaN point and haven't gone past it yet, + // just use that. + if (nextPointIdx >= idx) return; + + // We haven't found a non-NaN point yet or have moved past it, + // look towards the right to find a non-NaN point. + for (var j = idx; j < points.length; ++j) { + // Clear out a previously-found point (if any) since it's no longer + // valid, we shouldn't use it for interpolation anymore. + nextPoint = null; + if (!isNaN(points[j].yval) && points[j].yval !== null) { + nextPointIdx = j; + nextPoint = points[j]; + break; + } + } + }; + + for (var i = 0; i < points.length; ++i) { + var point = points[i]; + var xval = point.xval; + if (cumulativeYval[xval] === undefined) { + cumulativeYval[xval] = 0; + } + + var actualYval = point.yval; + if (isNaN(actualYval) || actualYval === null) { + // Interpolate/extend for stacking purposes if possible. + updateNextPoint(i); + if (prevPoint && nextPoint && fillMethod != 'none') { + // Use linear interpolation between prevPoint and nextPoint. + actualYval = prevPoint.yval + (nextPoint.yval - prevPoint.yval) * + ((xval - prevPoint.xval) / (nextPoint.xval - prevPoint.xval)); + } else if (prevPoint && fillMethod == 'all') { + actualYval = prevPoint.yval; + } else if (nextPoint && fillMethod == 'all') { + actualYval = nextPoint.yval; + } else { + actualYval = 0; + } + } else { + prevPoint = point; + } + + var stackedYval = cumulativeYval[xval]; + if (lastXval != xval) { + // If an x-value is repeated, we ignore the duplicates. + stackedYval += actualYval; + cumulativeYval[xval] = stackedYval; + } + lastXval = xval; + + point.yval_stacked = stackedYval; + + if (stackedYval > seriesExtremes[1]) { + seriesExtremes[1] = stackedYval; + } + if (stackedYval < seriesExtremes[0]) { + seriesExtremes[0] = stackedYval; + } + } +}; + + +/** * Loop over all fields and create datasets, calculating extreme y-values for * each series and extreme x-indices as we go. * @@ -2278,14 +2423,21 @@ Dygraph.prototype.predraw_ = function() { * extreme values "speculatively", i.e. without actually setting state on the * dygraph. * - * TODO(danvk): make this more of a true function - * @return [ datasets, seriesExtremes, boundaryIds ] + * @param {Array.)>>} rolledSeries, where + * rolledSeries[seriesIndex][row] = raw point, where + * seriesIndex is the column number starting with 1, and + * rawPoint is [x,y] or [x, [y, err]] or [x, [y, yminus, yplus]]. + * @param {?Array.} dateWindow [xmin, xmax] pair, or null. + * @return {{ + * points: Array.>, + * seriesExtremes: Array.>, + * boundaryIds: Array.}} * @private */ Dygraph.prototype.gatherDatasets_ = function(rolledSeries, dateWindow) { var boundaryIds = []; - var cumulative_y = []; // For stacked series. - var datasets = []; + var points = []; + var cumulativeYval = []; // For stacked series. var extremes = {}; // series name -> [low, high] var i, j, k; var errorBars = this.attr_("errorBars"); @@ -2306,21 +2458,13 @@ Dygraph.prototype.gatherDatasets_ = function(rolledSeries, dateWindow) { for (i = num_series; i >= 1; i--) { if (!this.visibility()[i - 1]) continue; - // Note: this copy _is_ necessary at the moment. - // If you remove it, it breaks zooming with error bars on. - // TODO(danvk): investigate further & write a test for this. - var series = []; - for (j = 0; j < rolledSeries[i].length; j++) { - series.push(rolledSeries[i][j]); - } - // Prune down to the desired range, if necessary (for zooming) // Because there can be lines going to points outside of the visible area, // we actually prune to visible points, plus one on either side. if (dateWindow) { + var series = rolledSeries[i]; var low = dateWindow[0]; var high = dateWindow[1]; - var pruned = []; // TODO(danvk): do binary search instead of linear search. // TODO(danvk): pass firstIdx and lastIdx directly to the renderer. @@ -2350,94 +2494,38 @@ Dygraph.prototype.gatherDatasets_ = function(rolledSeries, dateWindow) { isInvalidValue = isValueNull(series[correctedLastIdx]); } - boundaryIds[i-1] = [(firstIdx > 0) ? firstIdx - 1 : firstIdx, + boundaryIds[i-1] = [(firstIdx > 0) ? firstIdx - 1 : firstIdx, (lastIdx < series.length - 1) ? lastIdx + 1 : lastIdx]; if (correctedFirstIdx!==firstIdx) { - pruned.push(series[correctedFirstIdx]); - } - for (k = firstIdx; k <= lastIdx; k++) { - pruned.push(series[k]); + firstIdx = correctedFirstIdx; } if (correctedLastIdx !== lastIdx) { - pruned.push(series[correctedLastIdx]); + lastIdx = correctedLastIdx; } - - series = pruned; + // .slice's end is exclusive, we want to include lastIdx. + series = series.slice(firstIdx, lastIdx + 1); } else { + series = rolledSeries[i]; boundaryIds[i-1] = [0, series.length-1]; } + var seriesName = this.attr_("labels")[i]; var seriesExtremes = this.extremeValues_(series); - if (bars) { - for (j=0; j seriesExtremes[1]) { - seriesExtremes[1] = cumulative_y[x]; - } - if (cumulative_y[x] < seriesExtremes[0]) { - seriesExtremes[0] = cumulative_y[x]; - } - } + if (this.attr_("stackedGraph")) { + Dygraph.stackPoints_(seriesPoints, cumulativeYval, seriesExtremes, + this.attr_("stackedGraphNaNFill")); } - var seriesName = this.attr_("labels")[i]; extremes[seriesName] = seriesExtremes; - datasets[i] = series; - } - - // For stacked graphs, a NaN value for any point in the sum should create a - // clean gap in the graph. Back-propagate NaNs to all points at this X value. - if (this.attr_("stackedGraph")) { - for (k = datasets.length - 1; k >= 0; --k) { - // Use the first nonempty dataset to get X values. - if (!datasets[k]) continue; - for (j = 0; j < datasets[k].length; j++) { - var x = datasets[k][j][0]; - if (isNaN(cumulative_y[x])) { - // Set all Y values to NaN at that X value. - for (i = datasets.length - 1; i >= 0; i--) { - if (!datasets[i]) continue; - datasets[i][j][1] = NaN; - } - } - } - break; - } + points[i] = seriesPoints; } - return [ datasets, extremes, boundaryIds ]; + return { points: points, extremes: extremes, boundaryIds: boundaryIds }; }; /** @@ -2459,9 +2547,9 @@ Dygraph.prototype.drawGraph_ = function() { this.attrs_.pointSize = 0.5 * this.attr_('highlightCircleSize'); var packed = this.gatherDatasets_(this.rolledSeries_, this.dateWindow_); - var datasets = packed[0]; - var extremes = packed[1]; - this.boundaryIds_ = packed[2]; + var points = packed.points; + var extremes = packed.extremes; + this.boundaryIds_ = packed.boundaryIds; this.setIndexByName_ = {}; var labels = this.attr_("labels"); @@ -2469,10 +2557,10 @@ Dygraph.prototype.drawGraph_ = function() { this.setIndexByName_[labels[0]] = 0; } var dataIdx = 0; - for (var i = 1; i < datasets.length; i++) { + for (var i = 1; i < points.length; i++) { this.setIndexByName_[labels[i]] = i; if (!this.visibility()[i - 1]) continue; - this.layout_.addDataset(labels[i], datasets[i]); + this.layout_.addDataset(labels[i], points[i]); this.datasetIndex_[i] = dataIdx++; } @@ -2485,7 +2573,7 @@ Dygraph.prototype.drawGraph_ = function() { var tmp_zoomed_x = this.zoomed_x_; // Tell PlotKit to use this new data and render itself this.zoomed_x_ = tmp_zoomed_x; - this.layout_.evaluateWithError(); + this.layout_.evaluate(); this.renderGraph_(is_initial_draw); if (this.attr_("timingName")) { @@ -2812,6 +2900,13 @@ Dygraph.prototype.computeYAxisRanges_ = function(extremes) { * TODO(danvk): the "missing values" bit above doesn't seem right. * * @private + * @param {Array.)>>} rawData Input data. Rectangular + * grid of points, where rawData[row][0] is the X value for the row, + * and rawData[row][i] is the Y data for series #i. + * @param {number} i Series index, starting from 1. + * @param {boolean} logScale True if using logarithmic Y scale. + * @return {Array.)>} Series array, where + * series[row] = [x,y] or [x, [y, err]] or [x, [y, yplus, yminus]]. */ Dygraph.prototype.extractSeries_ = function(rawData, i, logScale) { // TODO(danvk): pre-allocate series here. -- 2.7.4