Data::Table - Data type related to database tables, spreadsheets, CSV/TSV files, HTML table displays, etc.
# some cool ways to use Table.pm
use Data::Table;
$header = ["name", "age"];
$data = [
["John", 20],
["Kate", 18],
["Mike", 23]
]
$t = new Data::Table($data, $header, 0); # Construct a table object with
# $data, $header, $type=0 (consider
# $data as the rows of the table).
print $t->csv; # Print out the table as a csv file.
$t = Data::Table::fromCSV("aaa.csv"); # Read a csv file into a table oject
print $t->html; # Diplay a 'portrait' HTML TABLE on web.
use DBI;
$dbh= DBI->connect("DBI:mysql:test", "test", "") or die $DBI::errstr;
my $minAge = 10;
$t = Data::Table::fromSQL($dbh, "select * from mytable where age >= ?", [$minAge]);
# Construct a table form an SQL
# database query.
$t->sort("age", 0, 0); # Sort by col 'age',numerical,descending
print $t->html2; # Print out a 'landscape' HTML Table.
$row = $t->delRow(2); # Delete the third row (index=2).
$t->addRow($row, 4); # Add the deleted row back as fifth row.
@rows = $t->delRows([0..2]); # Delete three rows (row 0 to 2).
$col = $t->delCol("age"); # Delete column 'age'.
$t->addCol($col, "age",2); # Add column 'age' as the third column
@cols = $t->delCols(["name","phone","ssn"]);
# Delete 3 columns at the same time.
$name = $t->elm(2,"name"); # Element access
$t2=$t->subTable([1, 3..4],['age', 'name']);
# Extract a sub-table
$t->rename("Entry", "New Entry"); # Rename column 'Entry' by 'New Entry'
$t->replace("Entry", [1..$t->nofRow()], "New Entry");
# Replace column 'Entry' by an array of
# numbers and rename it as 'New Entry'
$t->swap("age","ssn"); # Swap the positions of column 'age'
# with column 'ssn' in the table.
$t->colMap('name', sub {return uc}); # Map a function to a column
$t->sort('age',0,0,'name',1,0); # Sort table first by the numerical
# column 'age' and then by the
# string column 'name' in descending
# order
$t2=$t->match_pattern('$_->[0] =~ /^L/ && $_->[3]<0.2');
# Select the rows that matched the
# pattern specified
$t2=$t->match_string('John'); # Select the rows that matches 'John'
# in any column
$t2=$t->clone(); # Make a copy of the table. $t->rowMerge($t2); # Merge two tables $t->colMerge($t2);
This perl package uses perl5 objects to make it easy for manipulating spreadsheet data among disk files, database, and Web publishing.
A table object contains a header and a two-dimensional array of scalars. Three class methods Data::Table::fromCSV, Data::Table::fromTSV, and Data::Table::fromSQL allow users to create a table object from a CSV/TSV file or a database SQL selection in a snap.
Table methods provide basic access, add, delete row(s) or
column(s) operations, as well as more advanced sub-table
extraction, table sorting, record matching via keywords or patterns, table
merging, and web publishing. Data::Table class also provides a
straightforward interface to other popular Perl modules such as DBI and
GD::Graph.
The current version of Table.pm is available at http://www.geocities.com/easydatabase
We use Data::Table instead of Table, because Table.pm has already been used inside PerlQt module in CPAN.
A table object has three data members:
a reference to an array of array-references. It's basically a reference to a two-dimensional array.
a reference to a string array. The array contains all the column names.
1 means that @$data is an array of table columns (fields) (column-based); 0 means that @$data is an array of table rows (records) (row-based);
Row-based/Column-based are two internal implementations for a table object. E.g., if a spreadsheet consists of two columns lastname and age. In a row-based table, $data = [ ['Smith', 29], ['Dole', 32] ]. In a column-based table, $data = [ ['Smith', 'Dole'], [29, 32] ].
Two implementions have their pros and cons for different operations. Row-based implementation is better for sorting and pattern matching, while column-based one is better for adding/deleting/swapping columns.
Users only need to specify the implementation type of the table upon its creation via Data::Table::new, and can forget about it afterwards. Implementation type of a table should be considered volital, because methods switch table objects from one type into another internally. Be advised that row/column/element references gained via table::rowRef, table::rowRefs, table::colRef, table::colRefs, or table::elmRef may become stale after other method calls afterwards.
For those who want to inherit from the Data::Table class, internal method table::rotate is used to switch from one implementation type into another. There is an additional internal assistant data structure called colHash in our current implementation. This hash table stores all column names and their corresponding column index number as key-value pairs for fast conversion. This gives users an option to use column name wherever a column ID is expected, so that user don't have to use table::colIndex all the time. E.g., you may say $t->rename('oldColName', 'newColName') instead of $t->rename($t->colIndex('oldColName'), 'newColIdx').
contains a two-dimensional spreadsheet data.
contains all column names.
0 is row-based, 1 is column-based, describe the orientation of @$data.
see table::match_string and table::match_pattern
# =item $Data::Table::ID # #see Data::Table::fromSQL
Syntax: return_type method_name ( [ parameter [ = default_value ]] [, parameter [ = default_value ]] )
If method_name starts with table::, this is an instance method, it can be
used as $t->method( parameters ), where $t is a table
reference.
If method_name starts with Data::Table::, this is a class method, it should
be called as Data::Table::method, e.g., $t =
Data::Table::fromCSV(``filename.csv'').
Convensions for local variables:
colID: either a numerical column index or a column name; rowIdx: numerical row index; rowIDsRef: reference to an array of column IDs; rowIdcsRef: reference to an array of row indices; rowRef, colRef: reference to an array of scalars; data: ref_to_array_of_ref_to_array of data values; header: ref to array of column headers; table: a table object, a blessed reference.
create a new table. It returns a table object upon success, undef otherwise. $data: points to the spreadsheet data. $header: points to an array of column names. A column name must have at least one non-digit character. $type: 0 or 1 for row-based/column-based spreadsheet. $enforceCheck: 1/0 to turn on/off initial checking on the size of each row/column to make sure the data arguement indeed points to a valid structure.
create a new table, which is a subset of the original. It returns a table
object. $rowIdcsRef: points to an array of row indices. $colIDsRef: points
to an array of column IDs. The function make a copy of selected elements
from the original table. Undefined $rowIdcsRef or
$colIDsRef is interrpreted as all rows or all columns.
make a clone of the original. It return a table object, equivalent to table::subTable(undef,undef).
create a table from a CSV file. return a table object. $name: the CSV file name. $header: 0 or 1 to ignore/interrpret the first line in the file as column names, If it is set to 0, the default column names are ``col1'', ``col2'', ...
Same as Data::Table::fromCSV. However, this is an instant method (that's what 'i' stands for), which can be inheritated.
create a table from a TSV file. return a table object. $name: the TSV file name. $header: 0 or 1 to ignore/interrpret the first line in the file as column names, If it is set to 0, the default column names are ``col1'', ``col2'', ...
Note: read ``TSV FORMAT'' section for details.
Same as Data::Table::fromTSV. However, this is an instant method (that's what 'i' stands for), whic h can be inheritated.
create a table from the result of an SQL selection query. It returns a
table object upon success or undef otherwise. $dbh: a valid database
handler. Typically $dbh is obtained from DBI->connect, see
``Interface to Database'' or DBI.pm. $sql: an SQL query string. $vars:
optional reference to an array of variable values, required if
$sql contains '?'s which need to be replaced by the
corresponding variable values upon execution, see DBI.pm for details. Hint:
in MySQL, Data::Table::fromSQL($dbh, 'show tables from test') will also
create a valid table object.
Same as Data::Table::fromSQL. However, this is an instant method (that's what 'i' stands for), whic h can be inheritated.
translate a column name into its numerical position, the first column has index 0 as in as any perl array. return -1 for invalid column names.
return number of columns.
return number of rows.
return the value of a table element at [$rowIdx, $colID], undef if
$rowIdx or $colID is invalid.
return the reference to a table element at [$rowIdx, $colID], to allow
possible modification. It returns undef for invalid $rowIdx or
$colID.
return an array of column names.
return the implementation type of the table (row-based/column-based) at the time, be aware that the type of a table should be considered as volital during method calls.
return a string corresponding to the CSV representation of the table.
return a string corresponding to the TSV representation of the table.
Note: read ``TSV FORMAT'' section for details.
return a string corresponding to a 'Portrait/Lanscape'-style html-tagged
table. $colors: a reference to an array of three color strings, used for
backgrounds for table header, odd-row records, and even-row records,
respectively. A default color array (``#D4D4BF'',``#ECECE4'',``#CCCC99'')
will be used if $colors isn't defined.
$tag_tbl: a reference to a hash that specifies any legal attributes such as name, border, id, class, etc. for the TABLE tag.
$tag_tr: a reference to a hash that specifies any legal attributes for the TR tag.
$tag_th: a reference to a hash that specifies any legal attributes for the TH tag.
$tag_td: a reference to a hash that specifies any legal attributes for the TD tag.
Notice $tag_tr and $tag_th controls all the rows
and columns of the whole table. The keys of the hash are the attribute
names in these cases. However, $tag_td is column specific,
i.e., you should specify TD attributes for every column separately. The key
of %$tag_td are either column names or column indices, the
value is the full string to be inserted into the TD tag. E.g.,
$tag_td = {col3 => 'align=right valign=bottom} only change
the TD tag in ``col3'' to be <TD align=right valign=bottom>.
$portrait controls the layout of the table. The default is 1,
i.e., the table is shown in the ``Portrait'' style, like in Excel. 0 means
``Landscape''.
This method is depricated. It's here for compatibility. It now simple call
html method with $portrait = 0, see previous description.
return a string corresponding to a ``Landscape'' html-tagged table. This is useful to present a table with many columns, but very few entries. Check the above table::html for parameter descriptions.
modify the value of a table element at [$rowIdx, $colID] to a new value $val. It returns 1 upon success, undef otherwise.
add a new row ($rowRef points to the actual list of scalars), the new row
will be referred as $rowIdx as the result. E.g.,
addRow($aRow, 0) will put the new row as the very first row.
By default, it appends a row to the end. It returns 1 upon success, undef
otherwise.
delete a row at $rowIdx. It will the reference to the deleted row.
delete rows in @$rowIdcsRef. It will return an array of deleted rows upon success.
add a new column ($colRef points to the actual data), the new column will
be referred as $colName or $colIdx as the result.
E.g., addCol($aCol, 'newCol', 0) will put the new column as
the very first column. By default, append a row to the end. It will return
1 upon success or undef otherwise.
delete a column at $colID return the reference to the deleted
column.
delete a list of columns, pointed by $colIDsRef. It will return an array of deleted columns upon success.
return a reference to the row at $rowIdx upon success or undef
otherwise.
return a reference to array of row references upon success, undef otherwise.
return a copy of the row at $rowIdx upon success or undef
otherwise.
return a reference to a hash, which contains a copy of the row at $rowIdx, upon success or undef otherwise. The keys in the hash are column names, and the values are corresponding elements in that row. The hash is a copy, therefore modifying the hash values doesn't change the original table.
return a reference to the column at $colID upon success.
return a reference to array of column references upon success.
return a copy to the column at $colID upon success or undef
otherwise.
rename the column at $colID to a $newName (the
newName must be valid, and should not be idential to any other existing
column names). It returns 1 upon success or undef otherwise.
replace the column at $oldColID by the array pointed by
$newColRef, and renamed it to $newName. $newName is optional
if you don't want to rename the column. It returns 1 upon success or undef
otherwise.
swap two columns referred by $colID1 and $colID2. It returns 1
upon success or undef otherwise.
foreach element in column $colID, map a function $fun to it.
It returns 1 upon success or undef otherwise. This is a handy way to format
a column. E.g. if a column named URL contains URL strings,
colMap(``URL'', sub {``<a href='$_'>$_</a>''})
before html() will change each URL into a clickable hyper link
while displayed in a web browser.
sort a table in place. First sort by column $colID1 in
$order1 as $type1, then sort by $colID2 in
$order2 as $type2, ... $type is 0
for numerical and 1 for others; $order is 0 for ascending and
1 for descending; Sorting is done in the priority of colID1, colID2, ... It
returns 1 upon success or undef otherwise. Notice the table is rearranged
as a result! This is different from perl's list sort, which returns a
sorted copy while leave the original list untouched, the authors feel
inplace sorting is more natural.
return a new table consisting those rows evaluated to be true by
$pattern upon success or undef otherwise. Side effect:
@Data::Table::OK stores a true/false array for the original table rows.
Using it, users can find out what are the rows being selected/unselected.
In the $pattern string, a column element should be referred as
$_->[$colIndex]. E.g., match_pattern('$_->[0]>3
&& $_->[1]=~/^L') retrieve all the rows where its first column
is greater than 3 and second column starts with letter 'L'. Notice it only
takes colIndex, column names are not acceptable here!
return a new table consisting those rows contains string $s in
any of its fields upon success, undef otherwise. if
$caseIgnore evaluated to true, case will is be ignored
(s/$s/i).
Side effect: @Data::Table::OK stores a true/false array for the original
table rows. Using it, users can find out what are the rows being
selected/unselected. The $s string is actually treated as a
regular expression and applied to each row element, therefore one can
actually specify several keywords by saying, for instance,
match_string('One|Other').
mask is reference to an array, where elements are evaluated to be true or false. The size of the mask must be equal to the nofRow of the table. return a new table consisting those rows where the corresponding mask element is true (or false, when complement is set to true).
E.g., $t1=$tbl->match_string('keyword');
$t2=$tbl->rowMask(\@Data::Table::OK, 1) creates two new tables.
$t1 contains all rows match 'keyword', while $t2
contains all other rows.
Append all the rows in the table object $tbl to the original
rows. The merging table $tbl must have the same number of
columns as the original. It returns 1 upon success, undef otherwise. The
table object $tbl should not be used afterwards, since it
becomes part of the new table.
Append all the columns in table object $tbl to the original
columns. Table $tbl must have the same number of rows as the
original. It returns 1 upon success, undef otherwise. Table
$tbl should not be used afterwards, since it becomes part of
the new table.
Join two tables. The following join types are supported (defined by $type):
0: inner join 1: left outer join 2: right outer join 3: full outer join
$cols1 and $cols2 are references to array of
colIDs, where rows with the same elements in all listed columns are merged.
As the result table, columns listed in $cols2 are deleted,
before a new table is returned.
The implementation is hash-join, the running time should be linear with respect to the sum of number of rows in the two tables (assume both tables fit in memeory).
All internal methods are mainly implemented for used by other methods in the Table class. Users should avoid using them. Nevertheless, they are listed here for developers who would like to understand the code and may derive a new class from Data::Table.
convert the internal structure of a table between row-based and column-based. return 1 upon success, undef otherwise.
Encode an array of scalars into a CSV-formatted string.
Break a CSV encoded string to an array of scalars (check it out, we did it the cool way).
Encode an array of scalars into a TSV-formatted string.
Break a TSV encoded string to an array of scalars (check it out, we did it the cool way).
There is no standard for TSV format as far as we know. CSV format can't handle binary data very well, therefore, we choose the TSV format to overcome this limitation.
We define TSV based on MySQL convention.
"\0", "\n", "\t", "\r", "\b", "'", "\"", and "\\" are all escaped by '\' in the TSV file. (Warning: MySQL treats '\f' as 'f', and it's not escaped here) Undefined values are represented as '\N'.
Spreadsheet is a very generic type, therefore Data::Table class provides an easy interface between databases, web pages, CSV/TSV files, graphics packages, etc.
Here is a summary (partially repeat) of some classic usages of Data::Table.
use DBI;
$dbh= DBI->connect("DBI:mysql:test", "test", "") or die $DBI::errstr;
my $minAge = 10;
$t = Data::Table::fromSQL($dbh, "select * from mytable where age >= ?", [$minAge]);
print $t->html;
$t = fromCSV("mydata.csv");
$t->sort(1,1,0);
print $t->csv;
Same for TSV
use GD::Graph::points;
$graph = GD::Graph::points->new(400, 300);
$t2 = $t->match('$_->[1] > 20 && $_->[3] < 35.7');
my $gd = $graph->plot($t->colRefs([0,2]));
open(IMG, '>mygraph.png') or die $!;
binmode IMG;
print IMG $gd->png;
close IMG;
Copyright 1998-2000, Yingyao Zhou & Guangzhou Zou. All rights reserved.
It was first written by Zhou in 1998, significantly improved and maintained by Zou since 1999. The authors thank Tong Peng and Yongchuang Tao for valuable suggestions. We also thank those who kindly reported bugs, some of them are acknowledged in the ``Changes'' file.
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
Please send bug reports and comments to: easydatabase@yahoo.com. When sending bug reports, please provide the version of Table.pm, the version of Perl.
DBI, GD::Graph.