memorious

Crawler framework for documents and structured scrapers.

Usage

memorious [OPTIONS] COMMAND [ARGS]...

No arguments available

Name	Description	Required	Default
`-v, --version`	Show version	No	-
`--settings / --no-settings`	Show current settings [default: no-settings]	No	-
`--install-completion`	Install completion for the current shell.	No	-
`--show-completion`	Show completion for the current shell, to copy it or customize the installation.	No	-
`--help`	Show this message and exit.	No	-

Name	Description
`run`	Run a crawler from a YAML config file.
`worker`	Start the procrastinate worker to process...
`cancel`	Cancel execution of a crawler.
`flush`	Delete all data and tags generated by a...

Run a crawler from a YAML config file.

memorious run [OPTIONS] URI

Name	Description	Required
`URI`	URI or path to crawler YAML config file	Yes

Name	Description	Required	Default
`--src DIRECTORY`	Directory containing custom modules to add to Python path	No	-
`--continue-on-error`	Don't stop crawler execution on error	No	-
`--flush`	Delete all existing data before execution	No	-
`--help`	Show this message and exit.	No	-

Start the procrastinate worker to process crawler jobs.

memorious worker [OPTIONS]

No arguments available

Name	Description	Required	Default
`-c, --concurrency INTEGER`	Number of concurrent workers [default: 1]	No	-
`--help`	Show this message and exit.	No	-

Cancel execution of a crawler.

memorious cancel [OPTIONS] URI

Name	Description	Required
`URI`	URI or path to crawler YAML config file	Yes

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

Delete all data and tags generated by a crawler.

memorious flush [OPTIONS] URI

Name	Description	Required
`URI`	URI or path to crawler YAML config file	Yes

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-