From 48941eb37ffbd831fa82049f98a27ef34de87539 Mon Sep 17 00:00:00 2001 From: Christoph Ruegg Date: Sun, 9 Feb 2014 19:31:13 +0100 Subject: [PATCH] Docs: delimited files (CSV) --- MathNet.Numerics.sln | 1 + docs/content/CSV.fsx | 96 ++++++++++++++++++++++++++++ docs/tools/templates/template.cshtml | 3 +- 3 files changed, 99 insertions(+), 1 deletion(-) create mode 100644 docs/content/CSV.fsx diff --git a/MathNet.Numerics.sln b/MathNet.Numerics.sln index 184df47b..3a37d94b 100644 --- a/MathNet.Numerics.sln +++ b/MathNet.Numerics.sln @@ -40,6 +40,7 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Build", "Build", "{A4A66FA9 EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Docs", "Docs", "{039229DA-AFDA-48DB-B7FC-B064691DEE96}" ProjectSection(SolutionItems) = preProject + docs\content\CSV.fsx = docs\content\CSV.fsx docs\content\DescriptiveStatistics.fsx = docs\content\DescriptiveStatistics.fsx docs\content\Distance.fsx = docs\content\Distance.fsx docs\content\Functions.fsx = docs\content\Functions.fsx diff --git a/docs/content/CSV.fsx b/docs/content/CSV.fsx new file mode 100644 index 00000000..1df8e565 --- /dev/null +++ b/docs/content/CSV.fsx @@ -0,0 +1,96 @@ +(*** hide ***) +#I "../../out/lib/net40" +#r "MathNet.Numerics.dll" +#r "MathNet.Numerics.FSharp.dll" + +(** +Delimited Text Files (CSV & TSV) +================================ + +Likely the most common text file format for tabular data, CSV or TSV files store data as text +with one line per row and each column value separated by a comma or tabulator. +Such delimited text files are supported by virtually all software that deals with +tabular data in some way. + +Example: + + [lang=text] + A,B,C + 0.5,0.6,98.0 + 2.0,3.4,5.3 + +Unfortunately there are multiple ways to separate values and to deal with spacing around them. +CSV files typically use commas like in the example and TSV typically tabulators, but other separators +like semicolons or colons are common as well. Delimited files can optionally have headers on the first line +to describe the columns below (A, B and C in the example). + +Math.NET Numerics provides basic support for delimited files with the **MathNet.Numerics.Data.Text** package, +which is available on NuGet as separate package and not included in the basic distribution. + + +Reading a delimited file as matrix +---------------------------------- + +The `DelimitedReader` class provides static functions to read a matrix in delimited form: + +* **Read**: read from a TextReader. If you have your delimited data already in memory in a string, + you can use this method using a StringReader. +* **ReadStream**: read directly from a stream, e.g. a MemoryStream, FileStream or NetworkStream. +* **ReadFile**: read from a file, specified by the file system path. + +All these functions are generic in the data type of the matrix to be generated. +Only Double, Single, Complex and Complex32 are supported (C#: double, float; F#: float, float32). + +Example: + + [lang=csharp] + Matrix matrix = DelimitedReader.Read("data.csv", false, ",", true); + +In addition to the file path, stream or text reader as first argument, the behavior can be customized +with the following optional arguments: + +* **sparse**: Whether the the returned matrix should be constructed as sparse (true) or dense (false). + Default: false. +* **delimiter**: Number delimiter between numbers of the same line. Supports Regex groups. + Default: `\s` (white space). +* **hasHeaders**: Whether the first row contains column headers or not. + Default: false. +* **formatProvider**: The culture to use. It is often a good idea to use InvariantCulture here, + to make the format invariant from the local culture. + Default: null. + + +Writing a matrix as delimited file +---------------------------------- + +The dual to the delimiter above is the `DelimitedWriter` class that can serialize a matrix +to a delimited text file, stream or TextWriter. + +The static Write functions accept the following optional arguments, in addition to the matrix and target +as first two required arguments. + +* **delimiter**: Number delimiter to write between numbers of the same line. + Default: `,` (white space). +* **columnHeaders**: list of column header strings, or null if no headers should be written. + Default: null. +* **format**: The number format to use on each element. Default: null. +* **formatProvider**: The culture to use. It is often a good idea to use InvariantCulture here, + to make the format invariant from the local culture. + Default: null. + +Example: + + [lang=csharp] + DelimitedWriter.WriteFile(matrix, "data.csv"); + + +Alternatives +------------ + +The data extension packages also offer other ways to serialize a matrix to a binary stream or file. +Among others: + +* NIST MatrixMarket text files +* MATLAB mat files + +*) diff --git a/docs/tools/templates/template.cshtml b/docs/tools/templates/template.cshtml index 8de5c355..b5a8e07b 100644 --- a/docs/tools/templates/template.cshtml +++ b/docs/tools/templates/template.cshtml @@ -80,13 +80,14 @@
  • API Reference (new)
  • +
  • Delimited Text Files (CSV)
  • +
  • NIST MatrixMarket
  • FsLab & Deedle
  • Microsoft Excel
  • Intel MKL
  • numl.net machine learning
  • MATLAB
  • R-project
  • -
  • NIST MatrixMarket