docs/data.html

   1 <!DOCTYPE html>
   2 <html>
   3   <head>
   4     <meta http-equiv="X-UA-Compatible" content="IE=EmulateIE7; IE=EmulateIE9">
   5     <title>dygraphs input types</title>
   6     <style type="text/css">
   7       code { white-space: pre; border: 1px dashed black; display: block; }
   8       pre  { white-space: pre; border: 1px dashed black; }
   9       body { max-width: 800px; }
  10     </style>
  11   </head>
  12   <body>
  13     <h2>dygraphs Data Format</h2>
  14
  15     <p>When you create a Dygraph object, your code looks something like
  16     this:</p>
  17
  18     <code>
  19       g = new Dygraph(document.getElementById("div"),
  20                       <i>data</i>,
  21                       { <i>options</i> });
  22     </code>
  23
  24     <p>This document is about what you can put in the <i>data</i>
  25     parameter.</p>
  26
  27     <p>There are five types of input that dygraphs will accept:</p>
  28     <ol>
  29       <li><a href="#csv">CSV data</a>
  30       <li><a href="#url">URL</a>
  31       <li><a href="#array">array (native format)</a>
  32       <li><a href="#function">function</a>
  33       <li><a href="#datatable">DataTable</a>
  34     </ol>
  35
  36     <p>These are all discussed below. If you're trying to debug why your input
  37     won't parse, <b>check the JS error console</b>. dygraphs tries to log
  38     informative errors explaining what's wrong with your data, and these can
  39     often point you in the right direction.</p>
  40
  41     <p>There are several options which affect how your input data is
  42     interpreted. These are:
  43     <ul>
  44       <li> <i>xValueParser</i> affects CSV only.
  45       <li> <i>errorBars</i> affects all input types.
  46       <li> <i>customBars</i> affects all input types.
  47       <li> <i>fractions</i> affects all input types.
  48       <li> <i>labels</i> affects all input types.
  49     </ul>
  50     </p>
  51
  52     <a name="csv"><h3>CSV</h3>
  53     <p>Here's an example of what CSV data should look like:</p>
  54     <pre>
  55     Date,Series1,Series2
  56     2009/07/12,100,200  # comments are OK on data lines
  57     2009/07/19,150,201
  58     </pre>
  59
  60     <p>"CSV" is actually a bit of a misnomer: the data can be tab-delimited,
  61     too. The delimiter is set by the <i>delimiter</i> option. It default to ",".
  62     If no delimiter is found in the first row, it switches over to tab.</p>
  63
  64     <p>CSV parsing can be split into three parts: headers, x-value and
  65     y-values.</p>
  66
  67     <h4>Headers</h4>
  68     <p>If you don't specify the <i>labels</i> option, dygraphs will look at the
  69     first line of your CSV data to get the labels. If you see numbers for series
  70     labels when you hover over the dygraph, it's likely because your first line
  71     contains data but is being parsed as a label. The solution is to either add
  72     a header line or specify the labels like this:</p>
  73
  74     <code>
  75       new Dygraph(el,
  76                   "2009/07/12,100,200\n" +
  77                   "2009/07/19,150,201\n",
  78                   { labels: [ "Date", "Series1", "Series2" ] });
  79     </code>
  80
  81     <h4>x-values</h4>
  82     <p>Once the headers are parsed, dygraphs needs to determine what the type of
  83     the x values is. They're either dates or numbers. To make this
  84     determination, it looks at the first column of the first row ("2009/07/12"
  85     in the example above). Here's the heuristic: if it contains a '-' or a '/',
  86     or otherwise doesn't parse as a float, the it's a date. Otherwise, it's a
  87     number.</p>
  88
  89     <p>Once the type is determined, that doesn't mean all the values will parse
  90     correctly. The general rule is:<p>
  91
  92     <ul>
  93       <li>For dates, your strings have to be parseable by <i>Date.parse</i>.
  94       <li>For numbers, your strings have to be parseable by <i>parseFloat</i>.
  95     </ul>
  96
  97     <p>You can manually verify this using a JavaScript console. If a value
  98     doesn't parse, dygraphs will put a warning about it on your console. But
  99     beware: different browsers support different date formats!</p>
 100
 101     <p>Here are some valid date formats:</p>
 102     <ul>
 103       <li>2009-07-12</li>
 104       <li>2009/07/12</li>
 105       <li>2009/07/12 12</li>
 106       <li>2009/07/12 12:34</li>
 107       <li>2009/07/12 12:34:56</li>
 108     </ul>
 109
 110     <p>If you specify the <i>xValueParser</i> option, then all this detection is
 111     bypassed and your function is called instead. Your parser function takes in
 112     a string and needs to return a number. For dates/times, you should return
 113     milliseconds since epoch. You may also want to specify a few other options
 114     to make sure that everything gets displayed properly.<p>
 115
 116     <p>Here's code which parses a CSV file with unix timestamps in the first
 117     column:</p>
 118
 119     <code>
 120       new Dygraph(el,
 121                   "Date,Series1,Series2\n" +
 122                   "1247382000,100,200\n" +
 123                   "1247986800,150,201\n",
 124                   {
 125                     xValueFormatter: Dygraph.dateString_,
 126                     xValueParser: function(x) { return 1000*parseInt(x); },
 127                     xTicker: Dygraph.dateTicker
 128                   });
 129     </code>
 130
 131     <h4>y-values</h4>
 132     <p>Dependent (y-axis) values are simpler than x-values because they're
 133     always numbers. The complexity here comes from the various ways that you can
 134     specify the uncertainty in your measurements.<p>
 135
 136     <p>If your y-values are just numbers, then they need to be parseable by
 137     JavaScript's parseFloat function. Acceptable formats include:</p>
 138
 139     <ul>
 140       <li>12
 141       <li>-12
 142       <li>12.
 143       <li>12.3
 144       <li>1.24e+1
 145       <li>-1.24e+1
 146     </ul>
 147
 148     <p>If you have missing data, just leave the column blank (your CSV file will
 149     probably contain a ",," in it).</p>
 150
 151     <p>If your numbers have uncertainty associated with them, then there are
 152     three basic ways to express this: using fractions, standard deviations or
 153     explicit ranges.</p>
 154
 155     <h5>Fractions</h5>
 156     <p>If you specify the <i>fractions</i> option, then your data will all be
 157     interpreted as ratios between zero and one. This is often the case if you're
 158     plotting a percentage.</p>
 159
 160     <code>
 161       new Dygraph(el,
 162                   "X,Frac1,Frac2\n" +
 163                   "1,1/2,3/4\n"+
 164                   "2,1/3,2/3\n"+
 165                   "3,2/3,17/49\n"+
 166                   "4,25/30,100/200",
 167                   { fractions: true });
 168     </code>
 169
 170     <p>Why not just divide the fractions out yourself? There are two attractive
 171     reasons not to:</p>
 172
 173     <ul>
 174       <li>If you set both <i>fractions</i> and <i>errorBars</i>, then the
 175     denominator is interpreted as a sample size and dygraphs will plot <a
 176       href="http://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval">Wilson
 177       binomial proportion confidence intervals</a> around each point.
 178
 179       <li>If you set <i>showRoller</i>, then dygraphs will combine the values as
 180       fractions. If two point are <i>a/b</i> and <i>c/d</i>, it will plot
 181       <i>(a+b) / (c+d)</i> rather than <i>(a/b + c/d) / 2</i>, which is what
 182       you'd get if you divided the fractions through. This will also shrink the
 183       confidence intervals.</li>
 184     </ul>
 185
 186     <h5>Standard Deviations</h5>
 187     <p>Often you have a measurement and also a measure of its uncertainty: a
 188     standard deviation. If you specify the <i>errorBars</i> option, dygraphs
 189     will look for alternating value and standard deviation columns in your CSV
 190     data.  Here's what it should look like:</p>
 191
 192     <code>
 193       new Dygraph(el,
 194                   "X,Y1,Y2\n" +
 195                   "1,10,5,20,5\n" +
 196                   "2,12,5,22,5\n",
 197                   { errorBars: true });
 198     </code>
 199
 200     <p>The "5" values are standard deviations. When each point is plotted, a
 201     2-standard deviation region around it is shaded, resulting in a 95%
 202     confidence interval. If you want more or less confidence, you can set the
 203     <i>sigma</i> option to something other than 2.0.</p>
 204
 205     <p>When you roll data with standard deviations, dygraphs will plot the
 206     average of your values in each rolling period and the RMS value of your
 207     standard deviations: sqrt(std1 + std2 + std3 + ... + stdN)/N.</p>
 208
 209     <h5>Custom error bars</h5>
 210     <p>Sometimes your data has asymetric uncertainty or you want to specify
 211     something else with the error bars around a point. One example of this is
 212     the "temperatures" demo on the <a href="http://danvk.org/dygraphs">dygraphs
 213       home page.</a>, where the point is the daily average and the bars denote
 214     the low and high temperatures for the day.</p>
 215
 216     <p>To specify this format, set the <i>customBars</i> option. Your CSV values
 217     should each be three numbers separated by semicolons ("low;mid;high").
 218     Here's an example:</p>
 219
 220     <code>
 221       new Dygraph(el,
 222                   "X,Y1,Y2\n" +
 223                   "1,10;20;30,20;5;25\n" +
 224                   "2,10;25;35,20;10;25\n",
 225                   { customBars: true });
 226     </code>
 227
 228     <p>The middle value need not lie between the low and high values. If you set
 229     a rolling period, the three values will all be averaged independently.</p>
 230
 231
 232     <a name="url"><h3>URL</h3>
 233     <p>If you pass in a URL, dygraphs will issue an XMLHttpRequest for it and
 234     attempt to parse the returned data as CSV.
 235     </p>
 236
 237     <p><i>Common problems</i>. Make sure the URL is accessible and returns data
 238     in text format (as opposed to a CSV file with an HTML header). You can see
 239     what the response looks like by checking your JS console or by requesting
 240     the URL yourself.</p>
 241
 242
 243     <a name="array"><h3>Array (native format)</h3>
 244     <p>If you'll be constructing your data set from a server-side program (or
 245     from JavaScript) then you're better off producing an array than CSV data.
 246     This saves the cost of parsing the CSV data and also avoids common parser
 247     errors.</p>
 248
 249     <p>The downside is that it's harder to look at your data (you'll need to use
 250     a JS debugger) and that the data format is a bit less clear for values with
 251     uncertainties.</p>
 252
 253     <p>Here's an example of "native format":</p>
 254
 255     <code>
 256       new Dygraph(document.getElementById("graphdiv2"),
 257                   [
 258                     [1,10,100],
 259                     [2,20,80],
 260                     [3,50,60],
 261                     [4,70,80]
 262                   ],
 263                   {
 264                     labels: [ "x", "A", "B" ]
 265                   });
 266     </code>
 267
 268     <h4>Headers</h4>
 269     <p>Headers for native format must be specified via the <i>labels</i>
 270     option. There's no other way to set them.</p>
 271
 272     <h4>x-values</h4>
 273     <p>If you want your x-values to be dates, you'll need to use specify a Date
 274     object in the first column. Otherwise, specify a number. Here's a sample
 275     array with dates on the x-axis:</p>
 276
 277     <code>
 278       [
 279         [ new Date("2009/07/12"), 100, 200 ],
 280         [ new Date("2009/07/19"), 150, 220 ]
 281       ]
 282     </code>
 283
 284     <h4>y-values</h4>
 285     <p>You can specify <i>errorBars</i>, <i>fractions</i> or <i>customBars</i>
 286     with the array format. If you specify any of these, the values become arrays
 287     (rather than numbers). Here's what the format looks like for each one:</p>
 288
 289     <code>
 290       <i>errorBars</i>: [x, [value1, std1], [value2, std2], ...]
 291       <i>fractions</i>: [x, [num1, den1], [num2, den2], ...]
 292       <i>customBars</i>: [x, [low1, val1, high1], [low2, val2, high2], ...]
 293     </code>
 294
 295     <p>To specify missing data, set the value to null or NaN. You may not set a value
 296     inside an array to null or NaN. Use null or NaN instead of the entire array.
 297     The only difference between the two is when the option
 298     <a href="options.html#conectSeparatedPoints">connectSeparatedPoints</a>
 299     true. In that case, the gaps created by nulls are filled in, and gaps
 300     created by NaNs are preserved.
 301     </p>
 302
 303     <a name="function"><h3>Functions</h3>
 304
 305     <p>You can specify a function that returns any of the other types. If
 306     <i>x</i> is a valid piece of dygraphs input, then so is</p>
 307
 308     <code>
 309       function() { return x; }
 310     </code>
 311
 312     Functions can return strings, arrays, data tables, URLs, or any other data type.
 313
 314     <a name="datatable"><h3>DataTable</h3>
 315     <p>You can also specify a Google Visualization Library <a
 316       href="http://code.google.com/apis/visualization/documentation/reference.html#DataTable">DataTable</a>
 317     object as your input data. This lets you easily switch between dygraphs and
 318     other gviz visualizations such as the Annotated Timeline. It also lets you
 319     embed a Dygraph in a Google Spreadsheet.</p>
 320
 321     <p>You'll need to set your first column's type to one of "number", "date"
 322     or "datetime".</p>
 323
 324     <pre>
 325     DataTable TODO:
 326     - When to use Dygraph.GvizWrapper
 327     - how to specify fractions
 328     - how to specify missing data
 329     - how to specify value + std. dev.
 330     - how to specify [low, middle, high]
 331     - walkthrough of embedding a gadget in google docs/on a web page
 332     - walkthrough of using std. dev. in a spreadsheet chart
 333     </pre>
 334
 335   <!-- Google Analytics -->
 336 <script type="text/javascript">
 337   var _gaq = _gaq || [];
 338   _gaq.push(['_setAccount', 'UA-769809-2']);
 339   _gaq.push(['_trackPageview']);
 340   (function() {
 341     var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
 342     ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
 343     var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
 344   })();
 345 </script>
 346   </body>
 347 </html>