Module: ActiveSanction

Extended by:
T::Sig
Defined in:
lib/active_sanction/version.rb,
lib/active_sanction.rb,
lib/active_sanction/diff.rb,
lib/active_sanction/name.rb,
lib/active_sanction/sync.rb,
lib/active_sanction/error.rb,
lib/active_sanction/index.rb,
lib/active_sanction/query.rb,
lib/active_sanction/client.rb,
lib/active_sanction/doctor.rb,
lib/active_sanction/entity.rb,
lib/active_sanction/scorer.rb,
lib/active_sanction/address.rb,
lib/active_sanction/country.rb,
lib/active_sanction/fetcher.rb,
lib/active_sanction/matcher.rb,
lib/active_sanction/parsers.rb,
lib/active_sanction/sources.rb,
lib/active_sanction/storage.rb,
lib/active_sanction/subject.rb,
lib/active_sanction/testing.rb,
lib/active_sanction/rescreen.rb,
lib/active_sanction/snapshot.rb,
lib/active_sanction/phonetics.rb,
lib/active_sanction/identifier.rb,
lib/active_sanction/normalizer.rb,
lib/active_sanction/similarity.rb,
lib/active_sanction/validators.rb,
lib/active_sanction/deprecation.rb,
lib/active_sanction/diff/change.rb,
lib/active_sanction/http_client.rb,
lib/active_sanction/index/entry.rb,
lib/active_sanction/sync/report.rb,
lib/active_sanction/sync/result.rb,
lib/active_sanction/match_result.rb,
lib/active_sanction/parsers/join.rb,
lib/active_sanction/partial_date.rb,
lib/active_sanction/sources/base.rb,
lib/active_sanction/sources/ofac.rb,
lib/active_sanction/storage/base.rb,
lib/active_sanction/storage/meta.rb,
lib/active_sanction/configuration.rb,
lib/active_sanction/doctor/report.rb,
lib/active_sanction/index/builder.rb,
lib/active_sanction/payload_cache.rb,
lib/active_sanction/scorer/reason.rb,
lib/active_sanction/scorer/result.rb,
lib/active_sanction/doctor/checkup.rb,
lib/active_sanction/doctor/finding.rb,
lib/active_sanction/doctor/profile.rb,
lib/active_sanction/fetcher/result.rb,
lib/active_sanction/index/features.rb,
lib/active_sanction/parsers/format.rb,
lib/active_sanction/rescreen/alert.rb,
lib/active_sanction/scorer/subject.rb,
lib/active_sanction/scorer/weights.rb,
lib/active_sanction/sources/eu_fsf.rb,
lib/active_sanction/storage/memory.rb,
lib/active_sanction/index/candidate.rb,
lib/active_sanction/instrumentation.rb,
lib/active_sanction/normalizer/form.rb,
lib/active_sanction/snapshot/bundle.rb,
lib/active_sanction/sources/remarks.rb,
lib/active_sanction/validator_store.rb,
lib/active_sanction/doctor/diagnosis.rb,
lib/active_sanction/normalizer/cache.rb,
lib/active_sanction/sources/ofac_sdn.rb,
lib/active_sanction/scorer/name_score.rb,
lib/active_sanction/http_client/errors.rb,
lib/active_sanction/scorer/adjustments.rb,
lib/active_sanction/sources/definition.rb,
lib/active_sanction/parsers/spreadsheet.rb,
lib/active_sanction/parsers/xml_records.rb,
lib/active_sanction/partial_date/parser.rb,
lib/active_sanction/payload_cache/entry.rb,
lib/active_sanction/sources/canada_sema.rb,
lib/active_sanction/sources/ofac/record.rb,
lib/active_sanction/storage/file_system.rb,
lib/active_sanction/http_client/response.rb,
lib/active_sanction/parsers/column_shape.rb,
lib/active_sanction/similarity/token_set.rb,
lib/active_sanction/instrumentation/event.rb,
lib/active_sanction/normalizer/dictionary.rb,
lib/active_sanction/similarity/token_sort.rb,
lib/active_sanction/sources/eu_fsf/record.rb,
lib/active_sanction/storage/active_record.rb,
lib/active_sanction/payload_cache/checksum.rb,
lib/active_sanction/similarity/levenshtein.rb,
lib/active_sanction/snapshot/bundle/header.rb,
lib/active_sanction/sources/australia_dfat.rb,
lib/active_sanction/validator_store/memory.rb,
lib/active_sanction/parsers/delimited_table.rb,
lib/active_sanction/parsers/spreadsheet/row.rb,
lib/active_sanction/similarity/jaro_winkler.rb,
lib/active_sanction/snapshot/bundle/payload.rb,
lib/active_sanction/sources/un_consolidated.rb,
lib/active_sanction/snapshot/bundle/signature.rb,
lib/active_sanction/sources/ofac_consolidated.rb,
lib/active_sanction/sources/uk_sanctions_list.rb,
lib/active_sanction/storage/active_record/row.rb,
lib/active_sanction/parsers/spreadsheet/reader.rb,
lib/active_sanction/parsers/xml_records/reader.rb,
lib/active_sanction/parsers/xml_records/record.rb,
lib/active_sanction/phonetics/double_metaphone.rb,
lib/active_sanction/sources/canada_sema/record.rb,
lib/active_sanction/parsers/delimited_table/row.rb,
lib/active_sanction/parsers/spreadsheet/archive.rb,
lib/active_sanction/parsers/xml_records/builder.rb,
lib/active_sanction/sources/ofac/remarks_parser.rb,
lib/active_sanction/validator_store/file_system.rb,
lib/active_sanction/parsers/spreadsheet/workbook.rb,
lib/active_sanction/parsers/xml_records/backends.rb,
lib/active_sanction/storage/active_record/reader.rb,
lib/active_sanction/storage/active_record/writer.rb,
lib/active_sanction/instrumentation/notifications.rb,
lib/active_sanction/sources/australia_dfat/record.rb,
lib/active_sanction/normalizer/dictionary/stoplist.rb,
lib/active_sanction/parsers/delimited_table/reader.rb,
lib/active_sanction/sources/canada_sema/source_ref.rb,
lib/active_sanction/sources/un_consolidated/record.rb,
lib/active_sanction/sources/ofac_consolidated/record.rb,
lib/active_sanction/sources/uk_sanctions_list/record.rb,
lib/active_sanction/testing/storage_adapter_defaults.rb,
lib/active_sanction/parsers/xml_records/backends/rexml.rb,
lib/active_sanction/sources/ofac/remarks_parser/coverage.rb,
lib/generators/active_sanction/install/install_generator.rb,
lib/active_sanction/parsers/xml_records/backends/nokogiri.rb,
lib/active_sanction/sources/australia_dfat/published_date.rb,
lib/active_sanction/sources/ofac/remarks_parser/vocabulary.rb,
lib/active_sanction/sources/uk_sanctions_list/published_date.rb

Overview

How the storage conformance group builds the adapter it is testing.

In a module, and in a file of its own, for two reasons that are both about not defining a method twice. A group is free to override build_store in the customization block it passes to it_behaves_like -- which is how an adapter that cannot be built by .new alone gets held to the contract -- and defining a method over one the group already had would warn, since this suite runs with Ruby warnings on. Keeping it out of the shared example group's own file is the same rule one level up: the conformance spec loads that file again inside an RSpec sandbox, and a module reopened there would redefine whatever it holds.

Namespaced, because it ships: a bare StorageAdapterDefaults at the top level would be this gem putting a name into a host's global namespace to save itself two words.

Defined Under Namespace

Modules: Country, Deprecation, Error, Instrumentation, Parsers, Scorer, Sources, Storage, Testing Classes: Address, Client, Configuration, ConfigurationError, Diff, Doctor, Entity, FetchError, Identifier, IntegrityError, InvalidArgument, MatchResult, Matcher, MissingKey, Name, Normalizer, ParseError, PartialDate, Query, QueryError, Rescreen, Snapshot, SourceError, StorageError, Subject, Sync, UnsupportedError, ValidatorStore, Validators

Constant Summary collapse

VERSION =

The gem version, and what CHANGELOG.md is about. Moves under SemVer for a new source adapter, a storage fix or a documentation release -- none of which change what a name scores. MATCHER_VERSION, below, is the one that answers that question.

"1.1.1"
MATCHER_VERSION =

Which matching pipeline scored a decision, stamped onto every MatchResult and bumped whenever a change to the normalizer, the index, the similarity algorithms or the scorer could move a score.

Deliberately not VERSION. The gem version moves for a new source adapter, a storage fix, a documentation release -- none of which change what a name scores -- and an auditor asking "would this screening come out the same today?" needs the answer to that question rather than a release number that also answers several others. Its companions on the record are the weights and the snapshot checksum; between the three, a past decision is re-derivable.

"1"

Class Method Summary collapse

Class Method Details

.client ⇒ Client

The client the module-level calls answer through, built from the defaults on first use so that nothing has to remember to initialize it.

ActiveSanction.client.screen("Bosco Ntaganda")   # same as ActiveSanction.screen(...)

Everything below is sugar over this object. A process that needs two configurations at once -- a pinned list version for an audit re-run beside the current one for live traffic, one tenant's sources beside another's -- builds its own with Client.new and holds them itself; this one is what a script and the README quickstart use. See Client.

Returns:



81
82
83
 
# File 'lib/active_sanction.rb', line 81

def client
  CLIENT_LOCK.synchronize { @client ||= T.let(Client.new, T.nilable(Client)) }
end

.config ⇒ Configuration

The settings in force: whatever .with_configuration has installed on this fiber, or the default client's.

Frozen, because it belongs to a built client. configure is how it is changed, and it changes it by building a new client rather than by editing this one -- a settings object that could move underneath a running index is the thing Client exists to remove.

Returns:



93
 
# File 'lib/active_sanction.rb', line 93

def config = Thread.current[CONFIGURATION_KEY] || client.configuration

.configure(&block) ⇒ Configuration

The one entry point an application is expected to call at boot:

ActiveSanction.configure do |c|
c.user_agent = "my-app/1.0 (compliance@example.com)"
end

The block is handed a mutable copy of what is configured now, so settings accumulate across calls, and the copy is frozen into a new default client when the block returns. That replaces the held matcher, which is the behaviour a changed store or source list needs: an initializer that names a store must not leave a matcher behind that indexed a different one.

Configure at boot, before anything screens. Later is honoured from the next call and does not reach what has already happened -- names folded under the old dictionary are already in an index, and scores recorded under the old weights were recorded under the old weights.

Parameters:

Returns:



113
114
115
116
117
118
 
# File 'lib/active_sanction.rb', line 113

def configure(&block)
  settings = client.configuration.dup
  block.call(settings)
  CLIENT_LOCK.synchronize { @client = T.let(Client.new(configuration: settings), T.nilable(Client)) }
  settings
end

.diff(source = nil, **options) ⇒ Diff

What changed between two snapshots of one source:

diff = ActiveSanction.diff(:ofac_sdn, from: last_months_snapshot, to: todays_snapshot)
diff = ActiveSanction.diff(:ofac_sdn, from: last_months_snapshot)  # `to:` is what is stored now

diff.added     # => [Entity], newly listed
diff.removed   # => [Entity], delisted
diff.modified  # => [Diff::Change], amended, with the fields that moved
diff.changed   # => [Entity], what to re-screen a book of business against

So that re-screening runs against the eleven records that moved rather than against the whole list. A first sync -- from: nil -- is a baseline rather than a list of additions, and an amended record reports as one modification rather than as a delisting and a new listing. See Diff, which is where all of that is documented.

Parameters:

  • source (T.untyped) (defaults to: nil)
  • options (T.untyped)

Returns:



227
 
# File 'lib/active_sanction.rb', line 227

def diff(source = nil, **options) = T.unsafe(client).diff(source, **options)

.doctor(*sources, **options, &block) ⇒ Doctor::Report

Diagnoses whether a source's format has drifted -- fetching each list, parsing it, and comparing what it measures against the version that was stored at the last sync:

report = ActiveSanction.doctor                    # every configured source
report = ActiveSanction.doctor(:ofac_sdn)         # one
report = ActiveSanction.doctor(tolerance: 0.05)   # report smaller movements

report.ok?        # => false
report.findings   # => [Doctor::Finding, ...]
puts report
exit report.exit_code

The failure this exists for is the one a sync cannot see: a file that still parses cleanly and means something different. 19,321 entities carrying zero passports looks exactly as healthy as 19,321 carrying 23,429 if all anyone counts is records, and screening a passport number against the first returns a clean result for somebody who is on the list.

Nothing is written -- not the snapshot, not the payload cache, not the conditional-GET validators -- so a diagnosis can never be the reason a sync skipped a list that changed, and nothing here repairs anything. Deciding that a 40% drop in record count is a delisting wave rather than a broken parse is a judgment call this library does not make. See Doctor, which is where all of that is documented.

The block, if given, is called with each Doctor::Diagnosis as that source finishes.

Parameters:

  • sources (T.untyped)
  • options (T.untyped)
  • block (T.proc.params(diagnosis: Doctor::Diagnosis).void, nil)

Returns:



294
 
# File 'lib/active_sanction.rb', line 294

def doctor(*sources, **options, &block) = T.unsafe(client).doctor(*sources, **options, &block)

.export(source, **options) ⇒ Snapshot::Bundle::Header

Writes one stored list to a portable, optionally signed bundle file:

ActiveSanction.export(:ofac_sdn, to: "ofac_sdn.asb")
ActiveSanction.export(:ofac_sdn, to: "ofac_sdn.asb", sign_with: private_key)

One file, produced once, that another machine loads and screens against without reaching the publisher at all -- which is what an air-gapped installation needs, and what a deploy needs on the afternoon OFAC is down. See Snapshot::Bundle, and docs/bundle_format.md, which specifies the format well enough to be implemented outside Ruby.

Parameters:

  • source (T.untyped)
  • options (T.untyped)

Returns:



307
 
# File 'lib/active_sanction.rb', line 307

def export(source, **options) = T.unsafe(client).export(source, **options)

.import(path, **options) ⇒ Snapshot

Loads a bundle into the configured store and returns the snapshot it held:

snapshot = ActiveSanction.import("ofac_sdn.asb", verify_with: public_key)
snapshot.trusted?   # => true

A tampered bundle raises, a bundle signed by an unknown key raises something different, and both happen before anything is stored. See Client#import, which is where the one subtlety -- verification does not survive a write to disk -- is documented.

Parameters:

  • path (T.untyped)
  • options (T.untyped)

Returns:



320
 
# File 'lib/active_sanction.rb', line 320

def import(path, **options) = T.unsafe(client).import(path, **options)

.matcher ⇒ Matcher

The default client's matcher, built from its store on first use. See Client#matcher, which is where all of it is documented.

Returns:



167
 
# File 'lib/active_sanction.rb', line 167

def matcher = client.matcher

.reload! ⇒ T.self_type

Drops the shared matcher so the next screening call builds one over what is stored now. What a process calls after a sync -- a matcher is built once and never updated, which is what lets it be screened from many threads without a lock.

Returns:

  • (T.self_type) 


327
328
329
330
 
# File 'lib/active_sanction.rb', line 327

def reload!
  client.reload!
  self
end

.rescreen(subjects, diff:, **options, &block) ⇒ Array<Rescreen::Alert>

Applies a snapshot diff to a book of subjects, and returns the alerts:

book = [
ActiveSanction::Subject.new(id: "cust_1", name: "Bosco Ntaganda", date_of_birth: "1973"),
ActiveSanction::Subject.new(id: "cust_2", name: "Jane Miller")
]

alerts = ActiveSanction.rescreen(book, diff: diff, threshold: 75)

alerts.first.subject_id      # => "cust_1"
alerts.first.change          # => :newly_listed | :delisted | :details_changed
alerts.first.result          # => a full MatchResult, with its explanation
alerts.first.previous_score  # => what it scored against the old list version

Screening answers about a name; this answers about a book of business, and it is the step that turns a diff into an alert. The cost is the book times the handful of records that moved rather than the book times the whole corpus, which is what makes rescreening after every sync affordable -- an empty diff scores nothing at all.

A delisting raises an alert too: it is a change of status a compliance team has to record, and it is the one that lets a customer back through the door. The block, if given, is called with each alert as it is raised, so a large book streams past a small diff without accumulating anything. See Rescreen, which is where all of that is documented.

Parameters:

  • subjects (T.untyped)
  • diff (T.untyped)
  • options (T.untyped)
  • block (T.proc.params(alert: Rescreen::Alert).void, nil)

Returns:



258
259
260
 
# File 'lib/active_sanction.rb', line 258

def rescreen(subjects, diff:, **options, &block)
  T.unsafe(client).rescreen(subjects, diff: diff, **options, &block)
end

.reset! ⇒ T.self_type

Drops the default client and anything this fiber had installed, so the next call builds one from the defaults. What a suite runs between examples, and the reason a spec that configures a store does not leak it into the next one:

config.after { ActiveSanction.reset! }

It clears the fiber-local on the calling thread only; a thread that exited holding one has already taken it with it.

Returns:

  • (T.self_type) 


152
153
154
155
156
 
# File 'lib/active_sanction.rb', line 152

def reset!
  CLIENT_LOCK.synchronize { @client = T.let(nil, T.nilable(Client)) }
  Thread.current[CONFIGURATION_KEY] = nil
  self
end

.screen(query = nil, **overrides) ⇒ Array<MatchResult>

Screens one name against every configured list:

ActiveSanction.screen(name: "Bosco Ntaganda", type: :individual, threshold: 75)

Sugar over .matcher, which is where everything this does is documented.

Parameters:

  • query (T.untyped) (defaults to: nil)
  • overrides (T.untyped)

Returns:



175
 
# File 'lib/active_sanction.rb', line 175

def screen(query = nil, **overrides) = client.screen(query, **overrides)

.screen_all(queries, **overrides) ⇒ Array<Array<MatchResult>>

Screens a list of names, returning one array of results per query, in the order they were given. See Matcher#screen_all.

Parameters:

  • queries (T.untyped)
  • overrides (T.untyped)

Returns:



180
 
# File 'lib/active_sanction.rb', line 180

def screen_all(queries, **overrides) = client.screen_all(queries, **overrides)

.storage ⇒ Storage::Base

Where the default client's synced lists are read from. Gzipped JSON under storage_dir unless the application named its own -- see Configuration#storage.

Returns:



162
 
# File 'lib/active_sanction.rb', line 162

def storage = client.storage

.sync!(*sources, **options, &block) ⇒ Sync::Report

Fetches, parses and stores every configured list, and returns what each one did:

report = ActiveSanction.sync!                   # every configured source
report = ActiveSanction.sync!(:ofac_sdn)        # one
report = ActiveSanction.sync!(force: true)      # bypass conditional GET
report = ActiveSanction.sync!(concurrency: 3)   # fetch three publishers at once

report.failed?                                  # => false
report[:ofac_sdn].status                        # => :updated
exit report.exit_code                           # 1 if any source failed

A failing source does not raise and does not stop the others: it is captured into the report and its previous snapshot is kept, because yesterday's list with a visible age is safer than no list. See Sync, which is where all of that is documented, and Sync::Report.

The block, if given, is called with each Sync::Result as that source finishes -- the progress hook for a run that takes minutes.

Drops the shared matcher when any list changed, so the next screening call is answered by what was just synced. Not concurrent-safe against another sync of the same storage; see Client.

Parameters:

  • sources (T.untyped)
  • options (T.untyped)
  • block (T.proc.params(result: Sync::Result).void, nil)

Returns:



209
 
# File 'lib/active_sanction.rb', line 209

def sync!(*sources, **options, &block) = T.unsafe(client).sync!(*sources, **options, &block)

.with_configuration(configuration, &block) ⇒ T.untyped

Runs a block with configuration in force, so that everything reached from inside it reads those settings rather than the default client's. This is how a Client makes its own User-Agent, dictionary, XML backend and query defaults reach code that was written against the module -- the fetch layer, the normalizer, Query -- without every one of them having to be handed a configuration it does not otherwise want.

ActiveSanction.with_configuration(audit_client.configuration) { ... }

The one limit is the one every fiber-local has: a thread started inside the block does not inherit it, and starts from the default client's settings. Code that fans out has to reinstall the configuration in each worker, which is what Sync does.

Parameters:

  • configuration (Configuration)
  • block (T.proc.returns(T.untyped))

Returns:

  • (T.untyped) 


134
135
136
137
138
139
140
 
# File 'lib/active_sanction.rb', line 134

def with_configuration(configuration, &block)
  previous = Thread.current[CONFIGURATION_KEY]
  Thread.current[CONFIGURATION_KEY] = configuration
  block.call
ensure
  Thread.current[CONFIGURATION_KEY] = previous
end