|
|
|
@ -7,10 +7,9 @@ |
|
|
|
Delimited Text Files (CSV & TSV) |
|
|
|
================================ |
|
|
|
|
|
|
|
Likely the most common text file format for tabular data, CSV or TSV files store data as text |
|
|
|
with one line per row and each column value separated by a comma or tabulator. |
|
|
|
Such delimited text files are supported by virtually all software that deals with |
|
|
|
tabular data in some way. |
|
|
|
Likely the most common file format for tabular data, delimited files like CSV store data as text |
|
|
|
with one line per row and values within rows separated by a comma. |
|
|
|
Such text files are supported by virtually all software that deals with tabular data. |
|
|
|
|
|
|
|
Example: |
|
|
|
|
|
|
|
@ -19,69 +18,71 @@ Example: |
|
|
|
0.5,0.6,98.0 |
|
|
|
2.0,3.4,5.3 |
|
|
|
|
|
|
|
Unfortunately there are multiple ways to separate values and to deal with spacing around them. |
|
|
|
CSV files typically use commas like in the example and TSV typically tabulators, but other separators |
|
|
|
like semicolons or colons are common as well. Delimited files can optionally have headers on the first line |
|
|
|
to describe the columns below (A, B and C in the example). |
|
|
|
Unfortunately there is no universal standard on what character is used as separator and how |
|
|
|
individual values are formatted and escaped. CSV files traditionally use a comma as separator, but this |
|
|
|
causes problems e.g. in Germany where the comma is used as decimal point in numbers. The tabulator |
|
|
|
proves to be a useful alternative, usually denoted by using the TSV extension instead of CSV. |
|
|
|
Other separators like semicolons or colons are common as well. |
|
|
|
|
|
|
|
Math.NET Numerics provides basic support for delimited files with the **MathNet.Numerics.Data.Text** package, |
|
|
|
which is available on NuGet as separate package and not included in the basic distribution. |
|
|
|
|
|
|
|
|
|
|
|
Reading a delimited file as matrix |
|
|
|
---------------------------------- |
|
|
|
Reading a matrix from a delimited file |
|
|
|
-------------------------------------- |
|
|
|
|
|
|
|
The `DelimitedReader` class provides static functions to read a matrix in delimited form: |
|
|
|
The `DelimitedReader` class provides static functions to read a matrix from a file or string in delimited form. |
|
|
|
|
|
|
|
* **Read**: read from a TextReader. If you have your delimited data already in memory in a string, |
|
|
|
you can use this method using a StringReader. |
|
|
|
* **ReadStream**: read directly from a stream, e.g. a MemoryStream, FileStream or NetworkStream. |
|
|
|
* **ReadFile**: read from a file, specified by the file system path. |
|
|
|
|
|
|
|
All these functions are generic in the data type of the matrix to be generated. |
|
|
|
Only Double, Single, Complex and Complex32 are supported (C#: double, float; F#: float, float32). |
|
|
|
All these functions expect the data type of the matrix to be generated as generic type argument. |
|
|
|
Only Double, Single, Complex and Complex32 are supported. |
|
|
|
|
|
|
|
Example: |
|
|
|
|
|
|
|
[lang=csharp] |
|
|
|
Matrix<double> matrix = DelimitedReader.Read<double>("data.csv", false, ",", true); |
|
|
|
|
|
|
|
In addition to the file path, stream or text reader as first argument, the behavior can be customized |
|
|
|
with the following optional arguments: |
|
|
|
Unfortunately the lack of standard means that the parsing logic needs to be parametrized accordingly. |
|
|
|
There are ways to automatically profile the provided file to find out the correct parameters automatically, |
|
|
|
but for simplicity the Read functions expects those parameters explicitly as optional arguments: |
|
|
|
|
|
|
|
* **sparse**: Whether the the returned matrix should be constructed as sparse (true) or dense (false). |
|
|
|
Default: false. |
|
|
|
* **delimiter**: Number delimiter between numbers of the same line. Supports Regex groups. |
|
|
|
Default: `\s` (white space). |
|
|
|
* **hasHeaders**: Whether the first row contains column headers or not. |
|
|
|
* **hasHeaders**: Whether the first row contains column headers or not. If true, the first line will be skipped. |
|
|
|
Default: false. |
|
|
|
* **formatProvider**: The culture to use. It is often a good idea to use InvariantCulture here, |
|
|
|
to make the format invariant from the local culture. |
|
|
|
* **formatProvider**: The culture to use. It is often a good idea to use InvariantCulture, |
|
|
|
to make the format independent from the local culture. |
|
|
|
Default: null. |
|
|
|
|
|
|
|
|
|
|
|
Writing a matrix as delimited file |
|
|
|
---------------------------------- |
|
|
|
Writing a matrix to a delimited file |
|
|
|
------------------------------------ |
|
|
|
|
|
|
|
The dual to the delimiter above is the `DelimitedWriter` class that can serialize a matrix |
|
|
|
to a delimited text file, stream or TextWriter. |
|
|
|
|
|
|
|
The static Write functions accept the following optional arguments, in addition to the matrix and target |
|
|
|
as first two required arguments. |
|
|
|
The static Write functions accept the following optional arguments to control the output format: |
|
|
|
|
|
|
|
* **delimiter**: Number delimiter to write between numbers of the same line. |
|
|
|
Default: `,` (white space). |
|
|
|
Default: `\t` (tabulator). |
|
|
|
* **columnHeaders**: list of column header strings, or null if no headers should be written. |
|
|
|
Default: null. |
|
|
|
* **format**: The number format to use on each element. Default: null. |
|
|
|
* **formatProvider**: The culture to use. It is often a good idea to use InvariantCulture here, |
|
|
|
to make the format invariant from the local culture. |
|
|
|
* **format**: The number format to use on each element, similar to what can be provided to Double.ToString(). |
|
|
|
Default: null. |
|
|
|
* **formatProvider**: The culture to use. It is often a good idea to use InvariantCulture, |
|
|
|
to make the format independent from the local culture. |
|
|
|
Default: null. |
|
|
|
|
|
|
|
Example: |
|
|
|
|
|
|
|
[lang=csharp] |
|
|
|
DelimitedWriter.WriteFile(matrix, "data.csv"); |
|
|
|
DelimitedWriter.WriteFile(matrix, "data.csv", ","); |
|
|
|
|
|
|
|
|
|
|
|
Alternatives |
|
|
|
|