Class: ActiveSanction::Sources::UnConsolidated

Inherits:
Base
  • Object
show all
Extended by:
T::Sig
Defined in:
lib/active_sanction/sources/un_consolidated.rb,
lib/active_sanction/sources/un_consolidated/record.rb

Overview

The United Nations Security Council consolidated list: every individual and entity subject to a UN sanctions regime, in one document.

snapshot = ActiveSanction::Sources[:un_consolidated].new.sync

One file, two record shapes

https://scsanctions.un.org/resources/xml/en/consolidated.xml   2.2 MB

<INDIVIDUALS><INDIVIDUAL>   736 people
<ENTITIES><ENTITY>          275 organizations

Both are read in a single streaming pass. The file is small enough today to have been loaded whole without anyone noticing, which is exactly why it was not: the toolkit this adapter is written against (#15) is the one that will read OFAC's 126 MB advanced XML, and an adapter that quietly depended on holding the document would have to be rewritten then.

The trap in this list

QUALITY appears under both alias elements and means something different under each. Under <INDIVIDUAL_ALIAS> it grades the alias -- Good or Low -- which is a matching signal the scorer (#32) penalizes on. Under <ENTITY_ALIAS> it is not a grade at all; it is a.k.a. or f.k.a., which is an alias kind. Reading one as the other silently either throws away every entity's alias kind or grades 585 organization aliases on a scale that was never applied to them. Record maps them separately, and the counts above come from the published file rather than from a guess.

What this adapter does not do

INDIVIDUAL_PLACE_OF_BIRTH, GENDER, TITLE and DESIGNATION have no home in the canonical model. They are appended to remarks after a marker rather than dropped -- a place of birth is real screening signal, and losing it to keep a schema tidy is the wrong trade -- and are left for a later issue to structure if a matcher turns out to want them.

Instance Attribute Summary collapse

Attributes inherited from Base

#cache, #fetcher, #instrumenter, #logger

Instance Method Summary collapse

Methods inherited from Base

#authority, #column_shapes, #column_tallies, #file_key, #floors, #format, #fresh?, #inspect, #jurisdiction, #key, #parse_argument, published_remarks, #retrieve, #snapshot, #stale?, #sync, #url, #urls

Methods included from Definition

#authority, #declarations, #declared?, #declared_floors, #declared_urls, #file_key, #floor, #floors, #format, #jurisdiction, #key, key!, #licence_notice, #licence_url, #multi_url?, #to_h, #url, #urls

Constructor Details

#initialize(*args, **options) ⇒ void

Parameters:

  • args (T.untyped)
  • options (T.untyped) 


91
92
93
94
95
96
 
# File 'lib/active_sanction/sources/un_consolidated.rb', line 91

def initialize(*args, **options)
  super
  @warnings = T.let([], T::Array[Parsers::Warning])
  @unmapped = T.let([], T::Array[Parsers::Warning])
  @generated_at = T.let(nil, T.nilable(String))
end

Instance Attribute Details

#warnings ⇒ Array<Parsers::Warning> (readonly)

Records that could not be used. Read after #parse; sync orchestration (#34) reports them.

Returns:



88
89
90
 
# File 'lib/active_sanction/sources/un_consolidated.rb', line 88

def warnings
  @warnings
end

Instance Method Details

#parse(raw) ⇒ Array<Entity>

Parameters:

  • raw (T.untyped)

Returns:



99
100
101
102
103
104
105
 
# File 'lib/active_sanction/sources/un_consolidated.rb', line 99

def parse(raw)
  reader = LIST.read(raw)
  entities = build(reader)
  @generated_at = reader.root[GENERATED_AT]
  @warnings = reader.warnings + @unmapped
  entities
end

#source_version ⇒ String?

Returns:

  • (String, nil) 


108
 
# File 'lib/active_sanction/sources/un_consolidated.rb', line 108

def source_version = @generated_at || super