Store
PreservingLinker
Bases: Linker[StatementEntity]
A linker that hands back the canonical id a statement already carries.
Every backend writer stamps canonical_id from the store's linker
(nomenklatura.store.sql.SQLWriter.add_statement and its siblings), so a
store whose linker knows nothing collapses a resolved statement back onto
its entity id. Point the store at this linker instead and
preserve a statement before
handing it to the writer: the stamping then reproduces what the statement
already says.
Any other id maps to itself - the memory and leveldb writers also resolve entity-typed values through the linker for their inverted index, and those must not be answered with the subject's canonical id.
Source code in ftmq/store/base.py
Store
Bases: Store[Dataset, StatementEntity]
Feature add-ons to nomenklatura.store.Store
Source code in ftmq/store/base.py
218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 | |
scope
property
The effective read scope: the store's explicit dataset, or all
datasets present in the backend when it was opened without one.
__init__(dataset=None, linker=None, cast_types=True, **kwargs)
Initialize a store. This should be called via
get_store
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset
|
Dataset | str | None
|
A |
None
|
linker
|
Linker | None
|
A |
None
|
cast_types
|
bool
|
Normalize statement values on write (see
|
True
|
Source code in ftmq/store/base.py
casting_writer(writer)
Rebless a backend writer so it casts statement values on write (see
writer). A store that builds its
writer itself has to route it through here.
Source code in ftmq/store/base.py
get_scope()
iterate(dataset=None)
Iterate all the entities, optional filter for a dataset.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset
|
str | Dataset | None
|
|
None
|
Yields:
| Type | Description |
|---|---|
StatementEntities
|
Generator of |
Source code in ftmq/store/base.py
statements(dataset=None)
Iterate the raw statements in this store, as they are stored.
Unlike iterate, which reads
entities, this yields the stored rows unchanged - assembling an
entity rewrites entity-typed values to their canonical ids and
synthesizes its own id statement, so a dump taken that way would
carry statement ids that aren't in the store and would load back as new
rows instead of updating the existing ones. External statements are
included; the order is unspecified.
Only the SQL family of backends implements this.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset
|
str | Dataset | None
|
|
None
|
Yields:
| Type | Description |
|---|---|
Statements
|
Generator of |
Source code in ftmq/store/base.py
writer(*args, **kwargs)
The backend writer, normalizing statement values on the way in.
Values are cast into the canonical format of their property type (see
ftmq.statements); values that don't parse are
passed through unchanged (ftmq statements cast-types --drop-invalid
cleans those out of an existing dump). Disable with the store's
cast_types=False.
Source code in ftmq/store/base.py
View
Bases: View[Dataset, StatementEntity]
Feature add-ons to nomenklatura.store.base.View
Source code in ftmq/store/base.py
query(query=None)
Get the entities of a store, optionally filtered by a
Query object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
query
|
Query | None
|
The Query filter object |
None
|
Yields:
| Type | Description |
|---|---|
StatementEntities
|
Generator of |
Source code in ftmq/store/base.py
get_linker(uri)
cached
A read-only Linker: the merge decisions without the judgement history.
Use this where entities are only read (the api). The source is either a sql
database holding a nomenklatura resolver table, or an edge dump written
by Resolver.dump() / nomenklatura dump-resolver (json lines) - which
needs no database at all and can live anywhere anystore reads from.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
uri
|
Uri
|
A sql database uri, or a file-like uri of a json lines edge dump |
required |
Returns:
| Type | Description |
|---|---|
Linker[StatementEntity]
|
The linker. This is a cached object. |
Source code in ftmq/store/base.py
get_preserving_linker()
cached
The process-wide PreservingLinker.
Cached because get_store keys its cache on the linker object: a fresh
instance per call would cache (and keep) a fresh store per call. Only one
statement is in flight at a time, so writing statement streams into two
stores concurrently from one process is not supported.
Source code in ftmq/store/base.py
get_resolver(uri=None)
cached
The read/write Resolver backed by a sql resolver table.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
uri
|
str | None
|
A sql database uri. Anything else (or nothing) gets an ephemeral in-memory table. |
None
|
Returns:
| Type | Description |
|---|---|
Resolver[StatementEntity]
|
The resolver, with its decisions loaded. This is a cached object: it |
Resolver[StatementEntity]
|
keeps the cluster index in memory and nothing refreshes it, so a |
Resolver[StatementEntity]
|
process that needs another session's writes has to call |
Resolver[StatementEntity]
|
|