Browse Source

Docs: improve wording on delimited files docs

pull/197/head
Christoph Ruegg 13 years ago
parent
commit
d2eb0091d4
  1. 55
      docs/content/CSV.fsx

55
docs/content/CSV.fsx

@ -7,10 +7,9 @@
Delimited Text Files (CSV & TSV)
================================
Likely the most common text file format for tabular data, CSV or TSV files store data as text
with one line per row and each column value separated by a comma or tabulator.
Such delimited text files are supported by virtually all software that deals with
tabular data in some way.
Likely the most common file format for tabular data, delimited files like CSV store data as text
with one line per row and values within rows separated by a comma.
Such text files are supported by virtually all software that deals with tabular data.
Example:
@ -19,69 +18,71 @@ Example:
0.5,0.6,98.0
2.0,3.4,5.3
Unfortunately there are multiple ways to separate values and to deal with spacing around them.
CSV files typically use commas like in the example and TSV typically tabulators, but other separators
like semicolons or colons are common as well. Delimited files can optionally have headers on the first line
to describe the columns below (A, B and C in the example).
Unfortunately there is no universal standard on what character is used as separator and how
individual values are formatted and escaped. CSV files traditionally use a comma as separator, but this
causes problems e.g. in Germany where the comma is used as decimal point in numbers. The tabulator
proves to be a useful alternative, usually denoted by using the TSV extension instead of CSV.
Other separators like semicolons or colons are common as well.
Math.NET Numerics provides basic support for delimited files with the **MathNet.Numerics.Data.Text** package,
which is available on NuGet as separate package and not included in the basic distribution.
Reading a delimited file as matrix
----------------------------------
Reading a matrix from a delimited file
--------------------------------------
The `DelimitedReader` class provides static functions to read a matrix in delimited form:
The `DelimitedReader` class provides static functions to read a matrix from a file or string in delimited form.
* **Read**: read from a TextReader. If you have your delimited data already in memory in a string,
you can use this method using a StringReader.
* **ReadStream**: read directly from a stream, e.g. a MemoryStream, FileStream or NetworkStream.
* **ReadFile**: read from a file, specified by the file system path.
All these functions are generic in the data type of the matrix to be generated.
Only Double, Single, Complex and Complex32 are supported (C#: double, float; F#: float, float32).
All these functions expect the data type of the matrix to be generated as generic type argument.
Only Double, Single, Complex and Complex32 are supported.
Example:
[lang=csharp]
Matrix<double> matrix = DelimitedReader.Read<double>("data.csv", false, ",", true);
In addition to the file path, stream or text reader as first argument, the behavior can be customized
with the following optional arguments:
Unfortunately the lack of standard means that the parsing logic needs to be parametrized accordingly.
There are ways to automatically profile the provided file to find out the correct parameters automatically,
but for simplicity the Read functions expects those parameters explicitly as optional arguments:
* **sparse**: Whether the the returned matrix should be constructed as sparse (true) or dense (false).
Default: false.
* **delimiter**: Number delimiter between numbers of the same line. Supports Regex groups.
Default: `\s` (white space).
* **hasHeaders**: Whether the first row contains column headers or not.
* **hasHeaders**: Whether the first row contains column headers or not. If true, the first line will be skipped.
Default: false.
* **formatProvider**: The culture to use. It is often a good idea to use InvariantCulture here,
to make the format invariant from the local culture.
* **formatProvider**: The culture to use. It is often a good idea to use InvariantCulture,
to make the format independent from the local culture.
Default: null.
Writing a matrix as delimited file
----------------------------------
Writing a matrix to a delimited file
------------------------------------
The dual to the delimiter above is the `DelimitedWriter` class that can serialize a matrix
to a delimited text file, stream or TextWriter.
The static Write functions accept the following optional arguments, in addition to the matrix and target
as first two required arguments.
The static Write functions accept the following optional arguments to control the output format:
* **delimiter**: Number delimiter to write between numbers of the same line.
Default: `,` (white space).
Default: `\t` (tabulator).
* **columnHeaders**: list of column header strings, or null if no headers should be written.
Default: null.
* **format**: The number format to use on each element. Default: null.
* **formatProvider**: The culture to use. It is often a good idea to use InvariantCulture here,
to make the format invariant from the local culture.
* **format**: The number format to use on each element, similar to what can be provided to Double.ToString().
Default: null.
* **formatProvider**: The culture to use. It is often a good idea to use InvariantCulture,
to make the format independent from the local culture.
Default: null.
Example:
[lang=csharp]
DelimitedWriter.WriteFile(matrix, "data.csv");
DelimitedWriter.WriteFile(matrix, "data.csv", ",");
Alternatives

Loading…
Cancel
Save