Parent

Included Modules

Class Index [+]

Quicksearch

FasterCSV

This class provides a complete interface to CSV files and data. It offers tools to enable you to read and write to and from Strings or IO objects, as needed.

Reading

From a File

A Line at a Time

  FasterCSV.foreach("path/to/file.csv") do |row|
    # use row here...
  end

All at Once

  arr_of_arrs = FasterCSV.read("path/to/file.csv")

From a String

A Line at a Time

  FasterCSV.parse("CSV,data,String") do |row|
    # use row here...
  end

All at Once

  arr_of_arrs = FasterCSV.parse("CSV,data,String")

Writing

To a File

  FasterCSV.open("path/to/file.csv", "w") do |csv|
    csv << ["row", "of", "CSV", "data"]
    csv << ["another", "row"]
    # ...
  end

To a String

  csv_string = FasterCSV.generate do |csv|
    csv << ["row", "of", "CSV", "data"]
    csv << ["another", "row"]
    # ...
  end

Convert a Single Line

  csv_string = ["CSV", "data"].to_csv   # to CSV
  csv_array  = "CSV,String".parse_csv   # from CSV

Shortcut Interface

  FCSV             { |csv_out| csv_out << %w{my data here} }  # to $stdout
  FCSV(csv = "")   { |csv_str| csv_str << %w{my data here} }  # to a String
  FCSV($stderr)    { |csv_err| csv_err << %w{my data here} }  # to $stderr
  FCSV($stdin)     { |csv_in|  csv_in.each { |row| p row } }  # from $stdin

Advanced Usage

Wrap an IO Object

  csv = FCSV.new(io, options)
  # ... read (with gets() or each()) from and write (with <<) to csv here ...

Constants

VERSION

The version of the installed library.

FieldInfo

A FieldInfo Struct contains details about a field’s position in the data source it was read from. FasterCSV will pass this Struct to some blocks that make decisions based on field structure. See FasterCSV.convert_fields() for an example.

index

The zero-based index of the field in its row.

line

The line of the data source this row is from.

header

The header for the column, when available.

DateMatcher

A Regexp used to find and convert some common Date formats.

DateTimeMatcher

A Regexp used to find and convert some common DateTime formats.

Converters

This Hash holds the built-in converters of FasterCSV that can be accessed by name. You can select Converters with FasterCSV.convert() or through the options Hash passed to FasterCSV::new().

:integer

Converts any field Integer() accepts.

:float

Converts any field Float() accepts.

:numeric

A combination of :integer and :float.

:date

Converts any field Date::parse() accepts.

:date_time

Converts any field DateTime::parse() accepts.

:all

All built-in converters. A combination of :date_time and :numeric.

This Hash is intetionally left unfrozen and users should feel free to add values to it that can be accessed by all FasterCSV objects.

To add a combo field, the value should be an Array of names. Combo fields can be nested with other combo fields.

HeaderConverters

This Hash holds the built-in header converters of FasterCSV that can be accessed by name. You can select HeaderConverters with FasterCSV.header_convert() or through the options Hash passed to FasterCSV::new().

:downcase

Calls downcase() on the header String.

:symbol

The header String is downcased, spaces are replaced with underscores, non-word characters are dropped, and finally to_sym() is called.

This Hash is intetionally left unfrozen and users should feel free to add values to it that can be accessed by all FasterCSV objects.

To add a combo field, the value should be an Array of names. Combo fields can be nested with other combo fields.

DEFAULT_OPTIONS

The options used when no overrides are given by calling code. They are:

:col_sep

","

:row_sep

:auto

:quote_char

'"'

:converters

nil

:unconverted_fields

nil

:headers

false

:return_headers

false

:header_converters

nil

:skip_blanks

false

:force_quotes

false

Attributes

lineno[R]

The line number of the last row read from this file. Fields with nested line-end characters will not affect this count.

Public Class Methods

build_csv_interface() click to toggle source

This method will build a drop-in replacement for many of the standard CSV methods. It allows you to write code like:

  begin
    require "faster_csv"
    FasterCSV.build_csv_interface
  rescue LoadError
    require "csv"
  end
  # ... use CSV here ...

This is not a complete interface with completely identical behavior. However, it is intended to be close enough that you won’t notice the difference in most cases. CSV methods supported are:

Be warned that this interface is slower than vanilla FasterCSV due to the extra layer of method calls. Depending on usage, this can slow it down to near CSV speeds.

     # File lib/faster_csv.rb, line 874
874:     def self.build_csv_interface
875:       Object.const_set(:CSV, Class.new).class_eval do
876:         def self.foreach(path, rs = :auto, &block)  # :nodoc:
877:           FasterCSV.foreach(path, :row_sep => rs, &block)
878:         end
879: 
880:         def self.generate_line(row, fs = ",", rs = "")  # :nodoc:
881:           FasterCSV.generate_line(row, :col_sep => fs, :row_sep => rs)
882:         end
883: 
884:         def self.open(path, mode, fs = ",", rs = :auto, &block)  # :nodoc:
885:           if block and mode.include? "r"
886:             FasterCSV.open(path, mode, :col_sep => fs, :row_sep => rs) do |csv|
887:               csv.each(&block)
888:             end
889:           else
890:             FasterCSV.open(path, mode, :col_sep => fs, :row_sep => rs, &block)
891:           end
892:         end
893: 
894:         def self.parse(str_or_readable, fs = ",", rs = :auto, &block)  # :nodoc:
895:           FasterCSV.parse(str_or_readable, :col_sep => fs, :row_sep => rs, &block)
896:         end
897: 
898:         def self.parse_line(src, fs = ",", rs = :auto)  # :nodoc:
899:           FasterCSV.parse_line(src, :col_sep => fs, :row_sep => rs)
900:         end
901: 
902:         def self.readlines(path, rs = :auto)  # :nodoc:
903:           FasterCSV.readlines(path, :row_sep => rs)
904:         end
905:       end
906:     end
const_missing(*_) click to toggle source
    # File lib/faster_csv.rb, line 12
12:     def self.const_missing(*_)
13:       raise NotImplementedError, "Please switch to Ruby 1.9's standard CSV "  +
14:                                  "library.  It's FasterCSV plus support for " +
15:                                  "Ruby 1.9's m17n encoding engine."
16:     end
dump(ary_of_objs, io = "", options = Hash.new) click to toggle source

This method allows you to serialize an Array of Ruby objects to a String or File of CSV data. This is not as powerful as Marshal or YAML, but perhaps useful for spreadsheet and database interaction.

Out of the box, this method is intended to work with simple data objects or Structs. It will serialize a list of instance variables and/or Struct.members().

If you need need more complicated serialization, you can control the process by adding methods to the class to be serialized.

A class method csv_meta() is responsible for returning the first row of the document (as an Array). This row is considered to be a Hash of the form key_1,value_1,key_2,value_2,… FasterCSV::load() expects to find a class key with a value of the stringified class name and FasterCSV::dump() will create this, if you do not define this method. This method is only called on the first object of the Array.

The next method you can provide is an instance method called csv_headers(). This method is expected to return the second line of the document (again as an Array), which is to be used to give each column a header. By default, FasterCSV::load() will set an instance variable if the field header starts with an @ character or call send() passing the header as the method name and the field value as an argument. This method is only called on the first object of the Array.

Finally, you can provide an instance method called csv_dump(), which will be passed the headers. This should return an Array of fields that can be serialized for this object. This method is called once for every object in the Array.

The io parameter can be used to serialize to a File, and options can be anything FasterCSV::new() accepts.

     # File lib/faster_csv.rb, line 943
943:     def self.dump(ary_of_objs, io = "", options = Hash.new)
944:       obj_template = ary_of_objs.first
945: 
946:       csv = FasterCSV.new(io, options)
947: 
948:       # write meta information
949:       begin
950:         csv << obj_template.class.csv_meta
951:       rescue NoMethodError
952:         csv << [:class, obj_template.class]
953:       end
954: 
955:       # write headers
956:       begin
957:         headers = obj_template.csv_headers
958:       rescue NoMethodError
959:         headers = obj_template.instance_variables.sort
960:         if obj_template.class.ancestors.find { |cls| cls.to_s =~ /\AStruct\b/ }
961:           headers += obj_template.members.map { |mem| "#{mem}=" }.sort
962:         end
963:       end
964:       csv << headers
965: 
966:       # serialize each object
967:       ary_of_objs.each do |obj|
968:         begin
969:           csv << obj.csv_dump(headers)
970:         rescue NoMethodError
971:           csv << headers.map do |var|
972:             if var[0] == @@
973:               obj.instance_variable_get(var)
974:             else
975:               obj[var[0..2]]
976:             end
977:           end
978:         end
979:       end
980: 
981:       if io.is_a? String
982:         csv.string
983:       else
984:         csv.close
985:       end
986:     end
} click to toggle source

This method is a convenience for building Unix-like filters for CSV data. Each row is yielded to the provided block which can alter it as needed. After the block returns, the row is appended to output altered or not.

The input and output arguments can be anything FasterCSV::new() accepts (generally String or IO objects). If not given, they default to ARGF and $stdout.

The options parameter is also filtered down to FasterCSV::new() after some clever key parsing. Any key beginning with :in_ or :input_ will have that leading identifier stripped and will only be used in the options Hash for the input object. Keys starting with :out_ or :output_ affect only output. All other keys are assigned to both objects.

The :output_row_sep option defaults to $INPUT_RECORD_SEPARATOR ($/).

      # File lib/faster_csv.rb, line 1012
1012:     def self.filter(*args)
1013:       # parse options for input, output, or both
1014:       in_options, out_options = Hash.new, {:row_sep => $INPUT_RECORD_SEPARATOR}
1015:       if args.last.is_a? Hash
1016:         args.pop.each do |key, value|
1017:           case key.to_s
1018:           when /\Ain(?:put)?_(.+)\Z/
1019:             in_options[$1.to_sym] = value
1020:           when /\Aout(?:put)?_(.+)\Z/
1021:             out_options[$1.to_sym] = value
1022:           else
1023:             in_options[key]  = value
1024:             out_options[key] = value
1025:           end
1026:         end
1027:       end
1028:       # build input and output wrappers
1029:       input   = FasterCSV.new(args.shift || ARGF,    in_options)
1030:       output  = FasterCSV.new(args.shift || $stdout, out_options)
1031: 
1032:       # read, yield, write
1033:       input.each do |row|
1034:         yield row
1035:         output << row
1036:       end
1037:     end
foreach(path, options = Hash.new, &block) click to toggle source

This method is intended as the primary interface for reading CSV files. You pass a path and any options you wish to set for the read. Each row of file will be passed to the provided block in turn.

The options parameter can be anything FasterCSV::new() understands.

      # File lib/faster_csv.rb, line 1046
1046:     def self.foreach(path, options = Hash.new, &block)
1047:       open(path, "rb", options) do |csv|
1048:         csv.each(&block)
1049:       end
1050:     end
} click to toggle source

This method wraps a String you provide, or an empty default String, in a FasterCSV object which is passed to the provided block. You can use the block to append CSV rows to the String and when the block exits, the final String will be returned.

Note that a passed String is modfied by this method. Call dup() before passing if you need a new String.

The options parameter can be anthing FasterCSV::new() understands.

      # File lib/faster_csv.rb, line 1067
1067:     def self.generate(*args)
1068:       # add a default empty String, if none was given
1069:       if args.first.is_a? String
1070:         io = StringIO.new(args.shift)
1071:         io.seek(0, IO::SEEK_END)
1072:         args.unshift(io)
1073:       else
1074:         args.unshift("")
1075:       end
1076:       faster_csv = new(*args)  # wrap
1077:       yield faster_csv         # yield for appending
1078:       faster_csv.string        # return final String
1079:     end
generate_line(row, options = Hash.new) click to toggle source

This method is a shortcut for converting a single row (Array) into a CSV String.

The options parameter can be anthing FasterCSV::new() understands.

The :row_sep option defaults to $INPUT_RECORD_SEPARATOR ($/) when calling this method.

      # File lib/faster_csv.rb, line 1090
1090:     def self.generate_line(row, options = Hash.new)
1091:       options = {:row_sep => $INPUT_RECORD_SEPARATOR}.merge(options)
1092:       (new("", options) << row).string
1093:     end
instance(data = $stdout, options = Hash.new) click to toggle source

This method will return a FasterCSV instance, just like FasterCSV::new(), but the instance will be cached and returned for all future calls to this method for the same data object (tested by Object#object_id()) with the same options.

If a block is given, the instance is passed to the block and the return value becomes the return value of the block.

      # File lib/faster_csv.rb, line 1104
1104:     def self.instance(data = $stdout, options = Hash.new)
1105:       # create a _signature_ for this method call, data object and options
1106:       sig = [data.object_id] +
1107:             options.values_at(*DEFAULT_OPTIONS.keys.sort_by { |sym| sym.to_s })
1108: 
1109:       # fetch or create the instance for this signature
1110:       @@instances ||= Hash.new
1111:       instance    =   (@@instances[sig] ||= new(data, options))
1112: 
1113:       if block_given?
1114:         yield instance  # run block, if given, returning result
1115:       else
1116:         instance        # or return the instance
1117:       end
1118:     end
load(io_or_str, options = Hash.new) click to toggle source

This method is the reading counterpart to FasterCSV::dump(). See that method for a detailed description of the process.

You can customize loading by adding a class method called csv_load() which will be passed a Hash of meta information, an Array of headers, and an Array of fields for the object the method is expected to return.

Remember that all fields will be Strings after this load. If you need something else, use options to setup converters or provide a custom csv_load() implementation.

      # File lib/faster_csv.rb, line 1132
1132:     def self.load(io_or_str, options = Hash.new)
1133:       csv = FasterCSV.new(io_or_str, options)
1134: 
1135:       # load meta information
1136:       meta = Hash[*csv.shift]
1137:       cls  = meta["class"].split("::").inject(Object) do |c, const|
1138:         c.const_get(const)
1139:       end
1140: 
1141:       # load headers
1142:       headers = csv.shift
1143: 
1144:       # unserialize each object stored in the file
1145:       results = csv.inject(Array.new) do |all, row|
1146:         begin
1147:           obj = cls.csv_load(meta, headers, row)
1148:         rescue NoMethodError
1149:           obj = cls.allocate
1150:           headers.zip(row) do |name, value|
1151:             if name[0] == @@
1152:               obj.instance_variable_set(name, value)
1153:             else
1154:               obj.send(name, value)
1155:             end
1156:           end
1157:         end
1158:         all << obj
1159:       end
1160: 
1161:       csv.close unless io_or_str.is_a? String
1162: 
1163:       results
1164:     end
method_missing(*_) click to toggle source
    # File lib/faster_csv.rb, line 18
18:     def self.method_missing(*_)
19:       const_missing
20:     end
new(data, options = Hash.new) click to toggle source

This constructor will wrap either a String or IO object passed in data for reading and/or writing. In addition to the FasterCSV instance methods, several IO methods are delegated. (See FasterCSV::open() for a complete list.) If you pass a String for data, you can later retrieve it (after writing to it, for example) with FasterCSV.string().

Note that a wrapped String will be positioned at the beginning (for reading). If you want it at the end (for writing), use FasterCSV::generate(). If you want any other positioning, pass a preset StringIO object instead.

You may set any reading and/or writing preferences in the options Hash. Available options are:

:col_sep

The String placed between each field.

:row_sep

The String appended to the end of each row. This can be set to the special :auto setting, which requests that FasterCSV automatically discover this from the data. Auto-discovery reads ahead in the data looking for the next "r\n", "n", or "r" sequence. A sequence will be selected even if it occurs in a quoted field, assuming that you would have the same line endings there. If none of those sequences is found, or the stream is only available for output, the default $INPUT_RECORD_SEPARATOR ($/) is used. Obviously, discovery takes a little time. Set manually if speed is important. Also note that IO objects should be opened in binary mode on Windows if this feature will be used as the line-ending translation can cause problems with resetting the document position to where it was before the read ahead.

:quote_char

The character used to quote fields. This has to be a single character String. This is useful for application that incorrectly use ' as the quote character instead of the correct ". FasterCSV will always consider a double sequence this character to be an escaped quote.

:encoding

The encoding to use when parsing the file. Defaults to your $KCODE setting. Valid values: `n’ or `N’ for none, `e’ or `E’ for EUC, `s’ or `S’ for SJIS, and `u’ or `U’ for UTF-8 (see Regexp.new()).

:field_size_limit

This is a maximum size FasterCSV will read ahead looking for the closing quote for a field. (In truth, it reads to the first line ending beyond this size.) If a quote cannot be found within the limit FasterCSV will raise a MalformedCSVError, assuming the data is faulty. You can use this limit to prevent what are effectively DoS attacks on the parser. However, this limit can cause a legitimate parse to fail and thus is set to nil, or off, by default.

:converters

An Array of names from the Converters Hash and/or lambdas that handle custom conversion. A single converter doesn’t have to be in an Array.

:unconverted_fields

If set to true, an unconverted_fields() method will be added to all returned rows (Array or FasterCSV::Row) that will return the fields as they were before convertion. Note that :headers supplied by Array or String were not fields of the document and thus will have an empty Array attached.

:headers

If set to :first_row or true, the initial row of the CSV file will be treated as a row of headers. If set to an Array, the contents will be used as the headers. If set to a String, the String is run through a call of FasterCSV::parse_line() with the same :col_sep, :row_sep, and :quote_char as this instance to produce an Array of headers. This setting causes FasterCSV.shift() to return rows as FasterCSV::Row objects instead of Arrays and FasterCSV.read() to return FasterCSV::Table objects instead of an Array of Arrays.

:return_headers

When false, header rows are silently swallowed. If set to true, header rows are returned in a FasterCSV::Row object with identical headers and fields (save that the fields do not go through the converters).

:write_headers

When true and :headers is set, a header row will be added to the output. Note that if the table only contains header rows, :return_headers must also be set in order for a header row to be output.

:header_converters

Identical in functionality to :converters save that the conversions are only made to header rows.

:skip_blanks

When set to a true value, FasterCSV will skip over any rows with no content.

:force_quotes

When set to a true value, FasterCSV will quote all CSV fields it creates.

See FasterCSV::DEFAULT_OPTIONS for the default settings.

Options cannot be overriden in the instance methods for performance reasons, so be sure to set what you want here.

      # File lib/faster_csv.rb, line 1423
1423:     def initialize(data, options = Hash.new)
1424:       # build the options for this read/write
1425:       options = DEFAULT_OPTIONS.merge(options)
1426: 
1427:       # create the IO object we will read from
1428:       @io = if data.is_a? String then StringIO.new(data) else data end
1429: 
1430:       init_separators(options)
1431:       init_parsers(options)
1432:       init_converters(options)
1433:       init_headers(options)
1434: 
1435:       unless options.empty?
1436:         raise ArgumentError, "Unknown options:  #{options.keys.join(', ')}."
1437:       end
1438: 
1439:       # track our own lineno since IO gets confused about line-ends is CSV fields
1440:       @lineno = 0
1441:     end
new ) click to toggle source

This method opens an IO object, and wraps that with FasterCSV. This is intended as the primary interface for writing a CSV file.

You may pass any args Ruby’s open() understands followed by an optional Hash containing any options FasterCSV::new() understands.

This method works like Ruby’s open() call, in that it will pass a FasterCSV object to a provided block and close it when the block termminates, or it will return the FasterCSV object when no block is provided. (Note: This is different from the standard CSV library which passes rows to the block. Use FasterCSV::foreach() for that behavior.)

An opened FasterCSV object will delegate to many IO methods, for convenience. You may call:

  • binmode()

  • close()

  • close_read()

  • close_write()

  • closed?()

  • eof()

  • eof?()

  • fcntl()

  • fileno()

  • flush()

  • fsync()

  • ioctl()

  • isatty()

  • pid()

  • pos()

  • reopen()

  • seek()

  • stat()

  • sync()

  • sync=()

  • tell()

  • to_i()

  • to_io()

  • tty?()

      # File lib/faster_csv.rb, line 1211
1211:     def self.open(*args)
1212:       # find the +options+ Hash
1213:       options = if args.last.is_a? Hash then args.pop else Hash.new end
1214:       # default to a binary open mode
1215:       args << "rb" if args.size == 1
1216:       # wrap a File opened with the remaining +args+
1217:       csv     = new(File.open(*args), options)
1218: 
1219:       # handle blocks like Ruby's open(), not like the CSV library
1220:       if block_given?
1221:         begin
1222:           yield csv
1223:         ensure
1224:           csv.close
1225:         end
1226:       else
1227:         csv
1228:       end
1229:     end
new ) click to toggle source

This method can be used to easily parse CSV out of a String. You may either provide a block which will be called with each row of the String in turn, or just use the returned Array of Arrays (when no block is given).

You pass your str to read from, and an optional options Hash containing anything FasterCSV::new() understands.

      # File lib/faster_csv.rb, line 1243
1243:     def self.parse(*args, &block)
1244:       csv = new(*args)
1245:       if block.nil?  # slurp contents, if no block is given
1246:         begin
1247:           csv.read
1248:         ensure
1249:           csv.close
1250:         end
1251:       else           # or pass each row to a provided block
1252:         csv.each(&block)
1253:       end
1254:     end
parse_line(line, options = Hash.new) click to toggle source

This method is a shortcut for converting a single line of a CSV String into a into an Array. Note that if line contains multiple rows, anything beyond the first row is ignored.

The options parameter can be anything FasterCSV::new() understands.

      # File lib/faster_csv.rb, line 1263
1263:     def self.parse_line(line, options = Hash.new)
1264:       new(line, options).shift
1265:     end
read(path, options = Hash.new) click to toggle source

Use to slurp a CSV file into an Array of Arrays. Pass the path to the file and any options FasterCSV::new() understands.

      # File lib/faster_csv.rb, line 1271
1271:     def self.read(path, options = Hash.new)
1272:       open(path, "rb", options) { |csv| csv.read }
1273:     end
readlines(*args) click to toggle source

Alias for FasterCSV::read().

      # File lib/faster_csv.rb, line 1276
1276:     def self.readlines(*args)
1277:       read(*args)
1278:     end
table(path, options = Hash.new) click to toggle source

A shortcut for:

  FasterCSV.read( path, { :headers           => true,
                          :converters        => :numeric,
                          :header_converters => :symbol }.merge(options) )
      # File lib/faster_csv.rb, line 1287
1287:     def self.table(path, options = Hash.new)
1288:       read( path, { :headers           => true,
1289:                     :converters        => :numeric,
1290:                     :header_converters => :symbol }.merge(options) )
1291:     end

Public Instance Methods

<<(row) click to toggle source

The primary write method for wrapped Strings and IOs, row (an Array or FasterCSV::Row) is converted to CSV and appended to the data source. When a FasterCSV::Row is passed, only the row’s fields() are appended to the output.

The data source must be open for writing.

      # File lib/faster_csv.rb, line 1475
1475:     def <<(row)
1476:       # make sure headers have been assigned
1477:       if header_row? and [Array, String].include? @use_headers.class
1478:         parse_headers  # won't read data for Array or String
1479:         self << @headers if @write_headers
1480:       end
1481: 
1482:       # Handle FasterCSV::Row objects and Hashes
1483:       row = case row
1484:             when self.class::Row then row.fields
1485:             when Hash            then @headers.map { |header| row[header] }
1486:             else                      row
1487:             end
1488: 
1489:       @headers =  row if header_row?
1490:       @lineno  += 1
1491: 
1492:       @io << row.map(&@quote).join(@col_sep) + @row_sep  # quote and separate
1493: 
1494:       self  # for chaining
1495:     end
Also aliased as: add_row, puts
add_row(row) click to toggle source
Alias for: <<
} click to toggle source

You can use this method to install a FasterCSV::Converters built-in, or provide a block that handles a custom conversion.

If you provide a block that takes one argument, it will be passed the field and is expected to return the converted value or the field itself. If your block takes two arguments, it will also be passed a FieldInfo Struct, containing details about the field. Again, the block should return a converted field or the field itself.

      # File lib/faster_csv.rb, line 1514
1514:     def convert(name = nil, &converter)
1515:       add_converter(:converters, self.class::Converters, name, &converter)
1516:     end
each() click to toggle source

Yields each row of the data source in turn.

Support for Enumerable.

The data source must be open for reading.

      # File lib/faster_csv.rb, line 1545
1545:     def each
1546:       while row = shift
1547:         yield row
1548:       end
1549:     end
gets() click to toggle source
Alias for: shift
} click to toggle source

Identical to FasterCSV.convert(), but for header rows.

Note that this method must be called before header rows are read to have any effect.

      # File lib/faster_csv.rb, line 1529
1529:     def header_convert(name = nil, &converter)
1530:       add_converter( :header_converters,
1531:                      self.class::HeaderConverters,
1532:                      name,
1533:                      &converter )
1534:     end
header_row?() click to toggle source

Returns true if the next row read will be a header row.

      # File lib/faster_csv.rb, line 1567
1567:     def header_row?
1568:       @use_headers and @headers.nil?
1569:     end
inspect() click to toggle source

Returns a simplified description of the key FasterCSV attributes.

      # File lib/faster_csv.rb, line 1696
1696:     def inspect
1697:       str = "<##{self.class} io_type:"
1698:       # show type of wrapped IO
1699:       if    @io == $stdout then str << "$stdout"
1700:       elsif @io == $stdin  then str << "$stdin"
1701:       elsif @io == $stderr then str << "$stderr"
1702:       else                      str << @io.class.to_s
1703:       end
1704:       # show IO.path(), if available
1705:       if @io.respond_to?(:path) and (p = @io.path)
1706:         str << " io_path:#{p.inspect}"
1707:       end
1708:       # show other attributes
1709:       ] lineno     col_sep     row_sep
1710:           quote_char skip_blanks encoding ].each do |attr_name|
1711:         if a = instance_variable_get("@#{attr_name}")
1712:           str << " #{attr_name}:#{a.inspect}"
1713:         end
1714:       end
1715:       if @use_headers
1716:         str << " headers:#{(@headers || true).inspect}"
1717:       end
1718:       str << ">"
1719:     end
method_missing(*_) click to toggle source
    # File lib/faster_csv.rb, line 22
22:     def method_missing(*_)
23:       self.class.const_missing
24:     end
puts(row) click to toggle source
Alias for: <<
read() click to toggle source

Slurps the remaining rows and returns an Array of Arrays.

The data source must be open for reading.

      # File lib/faster_csv.rb, line 1556
1556:     def read
1557:       rows = to_a
1558:       if @use_headers
1559:         Table.new(rows)
1560:       else
1561:         rows
1562:       end
1563:     end
Also aliased as: readlines
readline() click to toggle source
Alias for: shift
readlines() click to toggle source
Alias for: read
rewind() click to toggle source

Rewinds the underlying IO object and resets FasterCSV’s lineno() counter.

      # File lib/faster_csv.rb, line 1458
1458:     def rewind
1459:       @headers = nil
1460:       @lineno  = 0
1461: 
1462:       @io.rewind
1463:     end
shift() click to toggle source

The primary read method for wrapped Strings and IOs, a single row is pulled from the data source, parsed and returned as an Array of fields (if header rows are not used) or a FasterCSV::Row (when header rows are used).

The data source must be open for reading.

      # File lib/faster_csv.rb, line 1578
1578:     def shift
1579:       #########################################################################
1580:       ### This method is purposefully kept a bit long as simple conditional ###
1581:       ### checks are faster than numerous (expensive) method calls.         ###
1582:       #########################################################################
1583: 
1584:       # handle headers not based on document content
1585:       if header_row? and @return_headers and
1586:          [Array, String].include? @use_headers.class
1587:         if @unconverted_fields
1588:           return add_unconverted_fields(parse_headers, Array.new)
1589:         else
1590:           return parse_headers
1591:         end
1592:       end
1593: 
1594:       # begin with a blank line, so we can always add to it
1595:       line = String.new
1596: 
1597:       # 
1598:       # it can take multiple calls to <tt>@io.gets()</tt> to get a full line,
1599:       # because of \r and/or \n characters embedded in quoted fields
1600:       # 
1601:       loop do
1602:         # add another read to the line
1603:         if read_line = @io.gets(@row_sep)
1604:          line += read_line
1605:         else
1606:          return nil
1607:         end
1608:         # copy the line so we can chop it up in parsing
1609:         parse =  line.dup
1610:         parse.sub!(@parsers[:line_end], "")
1611: 
1612:         # 
1613:         # I believe a blank line should be an <tt>Array.new</tt>, not 
1614:         # CSV's <tt>[nil]</tt>
1615:         # 
1616:         if parse.empty?
1617:           @lineno += 1
1618:           if @skip_blanks
1619:             line = ""
1620:             next
1621:           elsif @unconverted_fields
1622:             return add_unconverted_fields(Array.new, Array.new)
1623:           elsif @use_headers
1624:             return FasterCSV::Row.new(Array.new, Array.new)
1625:           else
1626:             return Array.new
1627:           end
1628:         end
1629: 
1630:         # parse the fields with a mix of String#split and regular expressions
1631:         csv           = Array.new
1632:         current_field = String.new
1633:         field_quotes  = 0
1634:         parse.split(@col_sep, 1).each do |match|
1635:           if current_field.empty? && match.count(@quote_and_newlines).zero?
1636:             csv           << (match.empty? ? nil : match)
1637:           elsif (current_field.empty? ? match[0] : current_field[0]) ==
1638:                 @quote_char[0]
1639:             current_field << match
1640:             field_quotes += match.count(@quote_char)
1641:             if field_quotes % 2 == 0
1642:               in_quotes = current_field[@parsers[:quoted_field], 1]
1643:               if !in_quotes || in_quotes[@parsers[:stray_quote]]
1644:                 raise MalformedCSVError,
1645:                       "Missing or stray quote in line #{lineno + 1}"
1646:               end
1647:               current_field = in_quotes
1648:               current_field.gsub!(@quote_char * 2, @quote_char) # unescape contents
1649:               csv           << current_field
1650:               current_field =  String.new
1651:               field_quotes  =  0
1652:             else # we found a quoted field that spans multiple lines
1653:               current_field << @col_sep
1654:             end
1655:           elsif match.count("\r\n").zero?
1656:             raise MalformedCSVError, "Illegal quoting in line #{lineno + 1}."
1657:           else
1658:             raise MalformedCSVError, "Unquoted fields do not allow " +
1659:                                      "\\r or \\n (line #{lineno + 1})."
1660:           end
1661:         end
1662: 
1663:         # if parse is empty?(), we found all the fields on the line...
1664:         if field_quotes % 2 == 0
1665:           @lineno += 1
1666: 
1667:           # save fields unconverted fields, if needed...
1668:           unconverted = csv.dup if @unconverted_fields
1669: 
1670:           # convert fields, if needed...
1671:           csv = convert_fields(csv) unless @use_headers or @converters.empty?
1672:           # parse out header rows and handle FasterCSV::Row conversions...
1673:           csv = parse_headers(csv)  if     @use_headers
1674: 
1675:           # inject unconverted fields and accessor, if requested...
1676:           if @unconverted_fields and not csv.respond_to? :unconverted_fields
1677:             add_unconverted_fields(csv, unconverted)
1678:           end
1679: 
1680:           # return the results
1681:           break csv
1682:         end
1683:         # if we're not empty?() but at eof?(), a quoted field wasn't closed...
1684:         if @io.eof?
1685:           raise MalformedCSVError, "Unclosed quoted field on line #{lineno + 1}."
1686:         elsif @field_size_limit and current_field.size >= @field_size_limit
1687:           raise MalformedCSVError, "Field size exceeded on line #{lineno + 1}."
1688:         end
1689:         # otherwise, we need to loop and pull some more data to complete the row
1690:       end
1691:     end
Also aliased as: gets, readline

Private Instance Methods

add_converter(var_name, const, name = nil, &converter) click to toggle source

The actual work method for adding converters, used by both FasterCSV.convert() and FasterCSV.header_convert().

This method requires the var_name of the instance variable to place the converters in, the const Hash to lookup named converters in, and the normal parameters of the FasterCSV.convert() and FasterCSV.header_convert() methods.

      # File lib/faster_csv.rb, line 1905
1905:     def add_converter(var_name, const, name = nil, &converter)
1906:       if name.nil?  # custom converter
1907:         instance_variable_get("@#{var_name}") << converter
1908:       else          # named converter
1909:         combo = const[name]
1910:         case combo
1911:         when Array  # combo converter
1912:           combo.each do |converter_name|
1913:             add_converter(var_name, const, converter_name)
1914:           end
1915:         else        # individual named converter
1916:           instance_variable_get("@#{var_name}") << combo
1917:         end
1918:       end
1919:     end
add_unconverted_fields(row, fields) click to toggle source

Thiw methods injects an instance variable unconverted_fields into row and an accessor method for it called unconverted_fields(). The variable is set to the contents of fields.

      # File lib/faster_csv.rb, line 1990
1990:     def add_unconverted_fields(row, fields)
1991:       class << row
1992:         attr_reader :unconverted_fields
1993:       end
convert_fields(fields, headers = false) click to toggle source

Processes fields with @converters, or @header_converters if headers is passed as true, returning the converted field set. Any converter that changes the field into something other than a String halts the pipeline of conversion for that field. This is primarily an efficiency shortcut.

      # File lib/faster_csv.rb, line 1928
1928:     def convert_fields(fields, headers = false)
1929:       # see if we are converting headers or fields
1930:       converters = headers ? @header_converters : @converters
1931: 
1932:       fields.enum_for(:each_with_index).map do |field, index|  # map_with_index
1933:         converters.each do |converter|
1934:           field = if converter.arity == 1  # straight field converter
1935:             converter[field]
1936:           else                             # FieldInfo converter
1937:             header = @use_headers && !headers ? @headers[index] : nil
1938:             converter[field, FieldInfo.new(index, lineno, header)]
1939:           end
1940:           break unless field.is_a? String  # short-curcuit pipeline for speed
1941:         end
1942:         field  # return final state of each field, converted or original
1943:       end
1944:     end
init_converters(options, field_name = :converters) click to toggle source

Loads any converters requested during construction.

If field_name is set :converters (the default) field converters are set. When field_name is :header_converters header converters are added instead.

The :unconverted_fields option is also actived for :converters calls, if requested.

      # File lib/faster_csv.rb, line 1855
1855:     def init_converters(options, field_name = :converters)
1856:       if field_name == :converters
1857:         @unconverted_fields = options.delete(:unconverted_fields)
1858:       end
1859: 
1860:       instance_variable_set("@#{field_name}", Array.new)
1861: 
1862:       # find the correct method to add the coverters
1863:       convert = method(field_name.to_s.sub(/ers\Z/, ""))
1864: 
1865:       # load converters
1866:       unless options[field_name].nil?
1867:         # allow a single converter not wrapped in an Array
1868:         unless options[field_name].is_a? Array
1869:           options[field_name] = [options[field_name]]
1870:         end
1871:         # load each converter...
1872:         options[field_name].each do |converter|
1873:           if converter.is_a? Proc  # custom code block
1874:             convert.call(&converter)
1875:           else                     # by name
1876:             convert.call(converter)
1877:           end
1878:         end
1879:       end
1880: 
1881:       options.delete(field_name)
1882:     end
init_headers(options) click to toggle source

Stores header row settings and loads header converters, if needed.

      # File lib/faster_csv.rb, line 1885
1885:     def init_headers(options)
1886:       @use_headers    = options.delete(:headers)
1887:       @return_headers = options.delete(:return_headers)
1888:       @write_headers  = options.delete(:write_headers)
1889: 
1890:       # headers must be delayed until shift(), in case they need a row of content
1891:       @headers = nil
1892: 
1893:       init_converters(options, :header_converters)
1894:     end
init_parsers(options) click to toggle source

Pre-compiles parsers and stores them by name for access during reads.

      # File lib/faster_csv.rb, line 1820
1820:     def init_parsers(options)
1821:       # store the parser behaviors
1822:       @skip_blanks      = options.delete(:skip_blanks)
1823:       @encoding         = options.delete(:encoding)  # nil will use $KCODE
1824:       @field_size_limit = options.delete(:field_size_limit)
1825: 
1826:       # prebuild Regexps for faster parsing
1827:       esc_col_sep = Regexp.escape(@col_sep)
1828:       esc_row_sep = Regexp.escape(@row_sep)
1829:       esc_quote   = Regexp.escape(@quote_char)
1830:       @parsers = {
1831:         :any_field    => Regexp.new( "[^#{esc_col_sep}]+",
1832:                                      Regexp::MULTILINE,
1833:                                      @encoding ),
1834:         :quoted_field => Regexp.new( "^#{esc_quote}(.*)#{esc_quote}$",
1835:                                      Regexp::MULTILINE,
1836:                                      @encoding ),
1837:         :stray_quote  => Regexp.new( "[^#{esc_quote}]#{esc_quote}[^#{esc_quote}]",
1838:                                      Regexp::MULTILINE,
1839:                                      @encoding ),
1840:         # safer than chomp!()
1841:         :line_end     => Regexp.new("#{esc_row_sep}\\z", nil, @encoding)
1842:       }
1843:     end
init_separators(options) click to toggle source

Stores the indicated separators for later use.

If auto-discovery was requested for @row_sep, this method will read ahead in the @io and try to find one. ARGF, STDIN, STDOUT, STDERR and any stream open for output only with a default @row_sep of $INPUT_RECORD_SEPARATOR ($/).

This method also establishes the quoting rules used for CSV output.

      # File lib/faster_csv.rb, line 1733
1733:     def init_separators(options)
1734:       # store the selected separators
1735:       @col_sep            = options.delete(:col_sep)
1736:       @row_sep            = options.delete(:row_sep)
1737:       @quote_char         = options.delete(:quote_char)
1738:       @quote_and_newlines = "\r\n#{@quote_char}"
1739: 
1740:       if @quote_char.length != 1
1741:         raise ArgumentError, ":quote_char has to be a single character String"
1742:       end
1743: 
1744:       # automatically discover row separator when requested
1745:       if @row_sep == :auto
1746:         begin
1747:           #
1748:           # remember where we were (pos() will raise an axception if @io is pipe
1749:           # or not opened for reading)
1750:           #
1751:           saved_pos = @io.pos
1752:           while @row_sep == :auto
1753:             #
1754:             # if we run out of data, it's probably a single line
1755:             # (ensure will set default value)
1756:             #
1757:             break if @io.eof?
1758: 
1759:             # read ahead a bit
1760:             sample =  @io.read(1024)
1761:             sample += @io.read(1) if sample[1..1] == "\r" and not @io.eof?
1762: 
1763:             # try to find a standard separator
1764:             if sample =~ /\r\n?|\n/
1765:               @row_sep = $&
1766:               break
1767:             end
1768:           end
1769: 
1770:           # tricky seek() clone to work around GzipReader's lack of seek()
1771:           @io.rewind
1772:           # reset back to the remembered position
1773:           while saved_pos > 1024  # avoid loading a lot of data into memory
1774:             @io.read(1024)
1775:             saved_pos -= 1024
1776:           end
1777:           @io.read(saved_pos) if saved_pos.nonzero?
1778:         rescue IOError         # not opened for reading
1779:           # do nothing:  ensure will set default
1780:         rescue NoMethodError   # Zlib::GzipWriter doesn't have eof?
1781:           # do nothing:  ensure will set default
1782:         rescue SystemCallError # pipe
1783:           # do nothing:  ensure will set default
1784:         ensure
1785:           #
1786:           # set default if we failed to detect
1787:           # (stream not opened for reading, a pipe, or a single line of data)
1788:           #
1789:           @row_sep = $INPUT_RECORD_SEPARATOR if @row_sep == :auto
1790:         end
1791:       end
1792: 
1793:       # establish quoting rules
1794:       do_quote = lambda do |field|
1795:         @quote_char                                      +
1796:         String(field).gsub(@quote_char, @quote_char * 2) +
1797:         @quote_char
1798:       end
1799:       @quote = if options.delete(:force_quotes)
1800:         do_quote
1801:       else
1802:         lambda do |field|
1803:           if field.nil?  # represent +nil+ fields as empty unquoted fields
1804:             ""
1805:           else
1806:             field = String(field)  # Stringify fields
1807:             # represent empty fields as empty quoted fields
1808:             if field.empty? or
1809:                field.count("\r\n#{@col_sep}#{@quote_char}").nonzero?
1810:               do_quote.call(field)
1811:             else
1812:               field  # unquoted field
1813:             end
1814:           end
1815:         end
1816:       end
1817:     end
parse_headers(row = nil) click to toggle source

This methods is used to turn a finished row into a FasterCSV::Row. Header rows are also dealt with here, either by returning a FasterCSV::Row with identical headers and fields (save that the fields do not go through the converters) or by reading past them to return a field row. Headers are also saved in @headers for use in future rows.

When nil, row is assumed to be a header row not based on an actual row of the stream.

      # File lib/faster_csv.rb, line 1956
1956:     def parse_headers(row = nil)
1957:       if @headers.nil?                # header row
1958:         @headers = case @use_headers  # save headers
1959:                    # Array of headers
1960:                    when Array  then @use_headers
1961:                    # CSV header String
1962:                    when String
1963:                      self.class.parse_line( @use_headers,
1964:                                             :col_sep    => @col_sep,
1965:                                             :row_sep    => @row_sep,
1966:                                             :quote_char => @quote_char )
1967:                    # first row is headers
1968:                    else             row
1969:                    end
1970: 
1971:         # prepare converted and unconverted copies
1972:         row      = @headers                       if row.nil?
1973:         @headers = convert_fields(@headers, true)
1974: 
1975:         if @return_headers                                     # return headers
1976:           return FasterCSV::Row.new(@headers, row, true)
1977:         elsif not [Array, String].include? @use_headers.class  # skip to field row
1978:           return shift
1979:         end
1980:       end
1981: 
1982:       FasterCSV::Row.new(@headers, convert_fields(row))  # field row
1983:     end

Disabled; run with --debug to generate this.

[Validate]

Generated with the Darkfish Rdoc Generator 1.1.6.