Class: ActiveSanction::Rescreen

Inherits:
Object
  • Object
show all
Extended by:
T::Sig
Defined in:
lib/active_sanction/rescreen.rb,
lib/active_sanction/rescreen/alert.rb

Overview

Who a list change affects. Screening answers about a name; this answers about a book of business.

book = [
ActiveSanction::Subject.new(id: "cust_1", name: "Bosco Ntaganda", date_of_birth: "1973"),
ActiveSanction::Subject.new(id: "cust_2", name: "Jane Miller")
]

diff   = ActiveSanction.diff(:ofac_sdn, from: yesterdays_snapshot)
alerts = ActiveSanction.rescreen(book, diff: diff, threshold: 75)

alerts.first.subject_id      # => "cust_1"
alerts.first.change          # => :newly_listed
alerts.first.result          # => a full MatchResult, with its explanation
alerts.first.previous_score  # => nil, or what it scored before

Why this is the primitive rather than a nightly full screen

Screening a customer once is a checkbox. The obligation is ongoing: somebody cleared last month may be listed today, and a delisting matters just as much, because it is the entry that lets a customer back through the door. That is recurring work, and the naive way to do it -- every subject against every record, every night -- costs the whole book times the whole corpus and stops being nightly somewhere around a few thousand customers.

A rescreen costs the whole book times the handful of records that moved. Diff (#35) computes what changed in the list; this computes who that change affects, and it is the step that turns a diff into an alert. On a typical day an OFAC diff is a few dozen records, so the index built here is a few dozen names rather than 46,000 -- see the numbers in the README.

It is built once and then only read

A Rescreen holds an index over the records the diff names, both versions of each amended one, the weights it scores with and the threshold it defaults to. All of it is fixed at construction and the object is frozen, so a large book is streamed past one of these, in batches, from as many threads as a host has:

rescreening = ActiveSanction::Rescreen.new(diff: diff, threshold: 75)
customers.find_each(batch_size: 1_000) do |batch|
rescreening.call(batch) { |alert| AlertRecord.create!(alert.to_h) }
end

Nothing about a book is held: subjects are read one at a time and only alerts are kept, so memory is a function of how much moved rather than of how many customers there are. The block is what makes even that bounded -- it is called with each alert as it is raised, and a host that writes them out as they arrive never accumulates the array at all.

It does not touch the matcher

A rescreen builds its own index, over the diff, and never reads the store or the client's matcher. That is the point: applying a diff to a book must not cost an index build over the whole corpus, and a host that has never screened anything in this process can rescreen without paying for one.

An empty diff does no work at all

A sync that changed nothing, or a first sync -- which is a baseline rather than a list of 19,015 additions, see Diff -- yields no alerts and scores nothing. Not one subject is folded. A host that syncs hourly and rescreens after each sync is paying for the hours that moved, which is what makes rescreening after every sync affordable.

What it does not do

It reports the changes; it does not remember them. There is no alert store, no deduplication against what was raised yesterday, and no disposition -- see the note on case management in the README. Two runs over the same diff produce the same alerts, which is the property that makes them re-derivable and the reason a host, not this library, owns the queue they go into.

It also cannot find what was already there. A subject who matched a record that did not change is not in a diff at all, and no rescreen will report them: the first screening run against a book is a deliberate full screen (Client#screen_all), and this is what keeps it current afterwards.

Defined Under Namespace

Classes: Alert

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(diff:, threshold: nil, weights: nil, candidate_limit: nil, backend: MatchResult::DEFAULT_BACKEND) ⇒ void

Parameters:

  • diff (T.untyped)
  • threshold (T.untyped) (defaults to: nil)
  • weights (T.untyped) (defaults to: nil)
  • candidate_limit (T.untyped) (defaults to: nil)
  • backend (T.untyped) (defaults to: MatchResult::DEFAULT_BACKEND) 


141
142
143
144
145
146
147
148
149
150
151
152
153
154
 
# File 'lib/active_sanction/rescreen.rb', line 141

def initialize(diff:, threshold: nil, weights: nil, candidate_limit: nil,
               backend: MatchResult::DEFAULT_BACKEND)
  @diff = T.let(diff!(diff), Diff)
  @threshold = T.let(threshold!(threshold), Float)
  @weights = T.let(Scorer::Weights.build(weights), Scorer::Weights)
  @candidate_limit = T.let(candidate_limit!(candidate_limit), Integer)
  @backend = T.let(backend.to_s.to_sym, Symbol)
  @versions = T.let(versions, T::Hash[String, T::Array[T.untyped]])
  @fields = T.let(amended, T::Hash[String, T::Array[Symbol]])
  # Both versions of an amended record are indexed, because a subject may
  # have matched only the alias that was taken away.
  @index = T.let(Index.build(@versions.values.flatten.compact), Index)
  freeze
end

Instance Attribute Details

#backend ⇒ Symbol (readonly)

Returns:

  • (Symbol) 


120
121
122
 
# File 'lib/active_sanction/rescreen.rb', line 120

def backend
  @backend
end

#candidate_limit ⇒ Integer (readonly)

How many names the index hands the scorer per subject. It does not bind on a typical diff -- a few dozen records cannot exceed it -- and is here for the day a publisher reissues a whole list under new ids.

Returns:

  • (Integer) 


117
118
119
 
# File 'lib/active_sanction/rescreen.rb', line 117

def candidate_limit
  @candidate_limit
end

#diff ⇒ Diff (readonly)

The diff being applied, which is what the alerts are about.

Returns:



100
101
102
 
# File 'lib/active_sanction/rescreen.rb', line 100

def diff
  @diff
end

#threshold ⇒ Float (readonly)

The lowest score worth an alert, for a subject that does not name its own. Read once, at construction, so a configuration changed mid-run cannot produce a book screened half one way.

Returns:

  • (Float) 


106
107
108
 
# File 'lib/active_sanction/rescreen.rb', line 106

def threshold
  @threshold
end

#weights ⇒ Scorer::Weights (readonly)

What each signal was worth for this run, and what every result it produces records.

Returns:



111
112
113
 
# File 'lib/active_sanction/rescreen.rb', line 111

def weights
  @weights
end

Class Method Details

.call(subjects, diff:, **options, &block) ⇒ Array<Alert>

Sugar, and what ActiveSanction.rescreen calls:

ActiveSanction::Rescreen.call(book, diff: diff, threshold: 75)

Builds a Rescreen and applies it once. A host streaming a book in batches builds one with .new and calls it per batch instead, so the index is built once rather than per batch.

Parameters:

  • subjects (T.untyped)
  • diff (T.untyped)
  • options (T.untyped)
  • block (T.proc.params(alert: Alert).void, nil)

Returns:



133
134
135
 
# File 'lib/active_sanction/rescreen.rb', line 133

def self.call(subjects, diff:, **options, &block)
  T.unsafe(self).new(diff: diff, **options).call(subjects, &block)
end

Instance Method Details

#call(subjects, &block) ⇒ Array<Alert>

The alerts this diff raises against this book, highest score first within each subject and in the order the subjects arrived:

rescreening.call(book)
rescreening.call(book) { |alert| queue.push(alert) }

subjects is anything that responds to each, so an Enumerator over a database cursor is streamed rather than materialized. Each entry is a Subject or the Hash one is built from.

Every alert in a run carries one screened_at, because a rescreening of a book against a new list version is a single event in an audit trail rather than ten thousand of them a microsecond apart. A host that calls this once per batch is running one event per batch, which is the honest description of what it did.

Parameters:

  • subjects (T.untyped)
  • block (T.proc.params(alert: Alert).void, nil)

Returns:



174
175
176
177
178
179
180
181
182
183
184
185
186
 
# File 'lib/active_sanction/rescreen.rb', line 174

def call(subjects, &block)
  return [] if diff.empty?

  screened_at = Time.now.utc
  alerts = T.let([], T::Array[Alert])
  each(subjects) do |value|
    found(Subject.build(value), screened_at).each do |alert|
      block&.call(alert)
      alerts << alert
    end
  end
  alerts
end

#empty? ⇒ Boolean

Nothing moved, so nothing can be affected. See the class comment.

Returns:

  • (Boolean) 


198
 
# File 'lib/active_sanction/rescreen.rb', line 198

def empty? = diff.empty?

#inspect ⇒ String

Returns:

  • (String) 


201
 
# File 'lib/active_sanction/rescreen.rb', line 201

def inspect = "#<#{self.class} #{source} #{size} changed records at #{threshold}>"

#size ⇒ Integer

How many records moved, which is what a run costs per subject.

Returns:

  • (Integer) 


194
 
# File 'lib/active_sanction/rescreen.rb', line 194

def size = diff.size

#source ⇒ Symbol

The list this run is about, taken from the diff.

Returns:

  • (Symbol) 


190
 
# File 'lib/active_sanction/rescreen.rb', line 190

def source = diff.source