Class: ActiveSanction::Parsers::XmlRecords

Inherits:
Object
  • Object
show all
Extended by:
T::Sig
Defined in:
lib/active_sanction/parsers/xml_records.rb,
lib/active_sanction/parsers/xml_records/reader.rb,
lib/active_sanction/parsers/xml_records/record.rb,
lib/active_sanction/parsers/xml_records/builder.rb,
lib/active_sanction/parsers/xml_records/backends.rb,
lib/active_sanction/parsers/xml_records/backends/rexml.rb,
lib/active_sanction/parsers/xml_records/backends/nokogiri.rb

Overview

Reads a record-oriented XML list -- the UN, Canada, the EU and the UK all publish one -- into records an adapter can map onto Entities.

A table is a description of the document, built once and reused for every sync; a Reader is one pass over one payload.

UN = ActiveSanction::Parsers::XmlRecords.new(records: %w[INDIVIDUAL ENTITY])

reader = UN.read(bytes)
reader.each do |record|
record.name                        # => "INDIVIDUAL"
record["FIRST_NAME"]               # => "ERIC"
record.values("NATIONALITY/VALUE") # => ["Chad"]
record.nodes("INDIVIDUAL_ALIAS")   # => [Record, ...]
end
reader.root["dateGenerated"]         # the publisher's own version marker

Streaming from day one, before anything needs it

Every list this gem launches with is small: the UN is 2.2 MB, Canada is 2.9 MB, and either would load into a DOM without anyone noticing. The one that is coming does not. OFAC's SDN_ADVANCED.XML is 126 MB, and a DOM design would meet it by being rewritten.

So the interface is record-at-a-time now, while it is free to be: the parser holds one record's depth on a stack and drops it as soon as the adapter is done with it. What that buys is not speed, it is that the adapter written against this today is the adapter that reads a 126 MB file later, unchanged.

Naming the records, and only the records

A document's scaffolding -- <CONSOLIDATED_LIST>, <INDIVIDUALS> -- is skipped entirely rather than being built into nodes nobody asked for. Naming several record elements is normal: the UN files people under <INDIVIDUAL> and organizations under <ENTITY>, in one document, and an adapter wants a single pass over both.

Namespaces

Element and attribute names are matched with any prefix removed, so a publisher adding an xmlns next quarter does not silently stop matching. See Backends.local_name for why the prefix is dropped rather than resolved.

Defined Under Namespace

Classes: MalformedDocument, Record

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(records:, null: nil, encoding: DEFAULT_ENCODING, backend: nil) ⇒ void

null: is here for the same reason DelimitedTable has it -- a publisher that writes a sentinel where it means nothing -- though the XML lists mostly use an empty element instead, which is already nil.

backend: overrides the configured default for this table alone. Most adapters should not pass it: which XML library parses a list is an installation's decision, not a list's. See Backends.

Parameters:

  • records (T.untyped)
  • null (T.untyped) (defaults to: nil)
  • encoding (T.untyped) (defaults to: DEFAULT_ENCODING)
  • backend (T.untyped) (defaults to: nil) 


91
92
93
94
95
96
97
 
# File 'lib/active_sanction/parsers/xml_records.rb', line 91

def initialize(records:, null: nil, encoding: DEFAULT_ENCODING, backend: nil)
  @records = T.let(records!(records), T::Set[String])
  @nulls = T.let(nulls!(null), T::Array[String])
  @encoding = T.let(encoding!(encoding), Encoding)
  @backend = T.let(backend&.to_sym, T.nilable(Symbol))
  freeze
end

Instance Attribute Details

#encoding ⇒ Encoding (readonly)

Returns:

  • (Encoding) 


81
82
83
 
# File 'lib/active_sanction/parsers/xml_records.rb', line 81

def encoding
  @encoding
end

#nulls ⇒ Array<String> (readonly)

Returns:

  • (Array<String>) 


78
79
80
 
# File 'lib/active_sanction/parsers/xml_records.rb', line 78

def nulls
  @nulls
end

#records ⇒ Set<String> (readonly)

A Set: record? is asked once per element in the document, which for the UN is roughly 30,000 times a pass.

Returns:

  • (Set<String>) 


75
76
77
 
# File 'lib/active_sanction/parsers/xml_records.rb', line 75

def records
  @records
end

Instance Method Details

#backend ⇒ T.untyped

Resolved per call rather than at construction, so a table built at class-definition time -- which is where an adapter builds it -- still honours an xml_backend set later in an initializer.

Returns:

  • (T.untyped) 


119
 
# File 'lib/active_sanction/parsers/xml_records.rb', line 119

def backend = Backends.resolve(@backend || ActiveSanction.config.xml_backend)

#inspect ⇒ String

Returns:

  • (String) 


122
123
124
 
# File 'lib/active_sanction/parsers/xml_records.rb', line 122

def inspect
  "#<#{self.class} records=#{record_names.join(", ")}#{" null=#{nulls.first.inspect}" if nulls.any?}>"
end

#read(payload) ⇒ Reader

A pass over one payload. Takes the bytes as a String, which is what Sources::Base hands #parse.

Parameters:

  • payload (T.untyped)

Returns:

  • (Reader) 


102
 
# File 'lib/active_sanction/parsers/xml_records.rb', line 102

def read(payload) = Reader.new(table: self, payload: payload)

#record?(name) ⇒ Boolean

Whether an element name starts a record. Asked by every backend for every element outside a record, so it stays a Set lookup.

Parameters:

  • name (String)

Returns:

  • (Boolean) 


107
 
# File 'lib/active_sanction/parsers/xml_records.rb', line 107

def record?(name) = records.include?(name)

#record_names ⇒ Array<String>

The record element names in declaration order, for a message or an inspect -- Set is the right shape to ask record? of and the wrong shape to print.

Returns:

  • (Array<String>) 


113
 
# File 'lib/active_sanction/parsers/xml_records.rb', line 113

def record_names = records.to_a