API Reference#
Databases#
- pydantic model volara.dbs.DB#
A base class for defining the common attributes and methods for all database types.
Show JSON schema
{ "title": "DB", "description": "A base class for defining the common attributes and methods for all\ndatabase types.", "type": "object", "properties": { "node_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Node Attrs" }, "edge_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Edge Attrs" }, "ndim": { "default": 3, "title": "Ndim", "type": "integer" } }, "additionalProperties": false }
- Config:
extra: str = forbid
- Fields:
-
field edge_attrs:
dict[str,str|int] |None= None# edge_attrs defines the custom edge attributes you may want to save between your super voxels. An example of some custom edge attrs is provided here:
edge_attrs={“z_aff”: “float”, “y_aff”: “float”, “x_aff”: “float”}
This adds a float columns with names [“{zyx}_affinity”] to the edges table in your graph db. You can later use these attributes by name to use these values in a blockwise task.
-
field ndim:
int= 3# The dimensionality of your spatial data. This is used to define the number of coordinates used to index your node locations.
-
field node_attrs:
dict[str,str|int] |None= None# node_attrs defines the custom node attributes you may want to save for your super voxels. An example of some custom node attrs is provided here:
node_attrs={“raw_intensity”: “float”}
This adds a float column with name “raw_intensity” to the nodes table in your graph db. You can later use this attribute by name to use this value in a blockwise task.
- abstract drop()#
Drop all nodes (supervoxels), edges, and metadata
- Return type:
None
- abstract drop_edges()#
Drop all edges between nodes (supervoxels) leaving the nodes and metadata in tact.
- Return type:
None
- init()#
Create database if it doesn’t exist yet.
- Return type:
None
- abstract open(mode='r')#
Return a funlib.persistence.graphs.GraphDB instance.
- Return type:
GraphDataBase
- property default_edge_attrs: dict[str, Any]#
The type definitions for the default edge attributes stored in our database.
Our DBs will always store edges between supervoxels with the following attributes:
distance: float:
The distance between center of masses of our super voxels
- property default_node_attrs: dict[str, Any]#
The type definitions for the default node attributes stored in our database.
Our DBs will always store supervoxels with the following attributes:
position: (float, float, float):
The center of mass of the supervoxel
size: int:
the number of voxels in this supervoxel
filtered: bool:
whether or not this fragment has been filtered out
- property graph_attrs: tuple[dict[str, Any], dict[str, Any]]#
Get all node and edge attributes including default and user provided attributes
- abstract property id: str#
A unique identifier for this databse.
- pydantic model volara.dbs.PostgreSQL#
A PostgreSQL database for storing and retrieving graph data.
Show JSON schema
{ "title": "PostgreSQL", "description": "A PostgreSQL database for storing and retrieving graph data.", "type": "object", "properties": { "node_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Node Attrs" }, "edge_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Edge Attrs" }, "ndim": { "default": 3, "title": "Ndim", "type": "integer" }, "db_type": { "const": "postgresql", "default": "postgresql", "title": "Db Type", "type": "string" }, "host": { "default": "localhost", "title": "Host", "type": "string" }, "name": { "default": "volara", "title": "Name", "type": "string" }, "user": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "User" }, "password": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Password" } }, "additionalProperties": false }
- Config:
extra: str = forbid
- Fields:
-
field db_type:
Literal['postgresql'] = 'postgresql'#
-
field host:
str= 'localhost'#
-
field name:
str= 'volara'#
-
field password:
str|None= None#
-
field user:
str|None= None#
- drop()#
Drop all nodes (supervoxels), edges, and metadata
- Return type:
None
- drop_edges()#
Drop all edges between nodes (supervoxels) leaving the nodes and metadata in tact.
- Return type:
None
- open(mode='r')#
Return a funlib.persistence.graphs.GraphDB instance.
- Return type:
PgSQLGraphDatabase
- property id: str#
A unique identifier for this databse.
- pydantic model volara.dbs.SQLite#
An SQLite database for storing and retrieving graph data.
Show JSON schema
{ "title": "SQLite", "description": "An SQLite database for storing and retrieving graph data.", "type": "object", "properties": { "node_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Node Attrs" }, "edge_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Edge Attrs" }, "ndim": { "default": 3, "title": "Ndim", "type": "integer" }, "db_type": { "const": "sqlite", "default": "sqlite", "title": "Db Type", "type": "string" }, "path": { "format": "path", "title": "Path", "type": "string" } }, "additionalProperties": false, "required": [ "path" ] }
- Config:
extra: str = forbid
- Fields:
-
field db_type:
Literal['sqlite'] = 'sqlite'# A literal used for pydantic serialization and deserialization of DB Union types.
-
field path:
Path[Required]# The path to the SQLite db file to use.
- drop()#
Drop all nodes (supervoxels), edges, and metadata
- Return type:
None
- drop_edges()#
Drop all edges between nodes (supervoxels) leaving the nodes and metadata in tact.
- Return type:
None
- open(mode='r')#
Return a funlib.persistence.graphs.GraphDB instance.
- Return type:
SQLiteGraphDataBase
- property id: str#
A unique identifier for this databse.
Datasets#
- pydantic model volara.datasets.Affs#
Represents a dataset containing affinities. Requires the inclusion of the neighborhood for these affinities.
Show JSON schema
{ "title": "Affs", "description": "Represents a dataset containing affinities.\nRequires the inclusion of the neighborhood for these\naffinities.", "type": "object", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "affs", "default": "affs", "title": "Dataset Type", "type": "string" }, "neighborhood": { "anyOf": [ { "items": { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Neighborhood" } }, "additionalProperties": false, "required": [ "store" ] }
- Config:
extra: str = forbid
- Fields:
-
field dataset_type:
Literal['affs'] = 'affs'#
-
field neighborhood:
list[PydanticCoordinate] |None= None#
- model_post_init(context)#
Override this method to perform additional initialization after __init__ and model_construct. This is useful if you want to do some validation that requires the entire model to be initialized.
- property attrs#
- pydantic model volara.datasets.Dataset#
A Dataset base class that defines the common attributes and methods for all dataset types.
Show JSON schema
{ "title": "Dataset", "description": "A Dataset base class that defines the common attributes and methods\nfor all dataset types.", "type": "object", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" } }, "additionalProperties": false, "required": [ "store" ] }
- Config:
extra: str = forbid
- Fields:
-
field axis_names:
list[str] |None= None#
-
field offset:
PydanticCoordinate|None= None#
-
field store:
str|Path[Required]#
-
field units:
list[str] |None= None#
-
field voxel_size:
PydanticCoordinate|None= None#
- array(mode='r')#
- Return type:
Array
- drop()#
Delete this dataset
- Return type:
None
- prepare(shape, chunk_shape, offset, voxel_size, units, axis_names, types, dtype)#
- Return type:
None
- abstract property attrs#
- property name: str#
A name for this dataset. Often it is simply the name of the path provided as the store. We use it to differentiate between multiple runs of the same blockwise task on different data.
- pydantic model volara.datasets.LSD#
Represents a dataset containing local shape descriptors.
Show JSON schema
{ "title": "LSD", "description": "Represents a dataset containing local shape descriptors.", "type": "object", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "lsd", "default": "lsd", "title": "Dataset Type", "type": "string" } }, "additionalProperties": false, "required": [ "store" ] }
- Config:
extra: str = forbid
- Fields:
-
field dataset_type:
Literal['lsd'] = 'lsd'#
- property attrs#
- pydantic model volara.datasets.Labels#
Represents an integer label dataset.
Show JSON schema
{ "title": "Labels", "description": "Represents an integer label dataset.", "type": "object", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "labels", "default": "labels", "title": "Dataset Type", "type": "string" } }, "additionalProperties": false, "required": [ "store" ] }
- Config:
extra: str = forbid
- Fields:
-
field dataset_type:
Literal['labels'] = 'labels'#
- property attrs#
- pydantic model volara.datasets.Raw#
Represents a dataset containing raw intensities. Has support for sampling specific channels, normalizing with provided scale and shifting, or reading in normalization bounds from OMERO metadata.
Show JSON schema
{ "title": "Raw", "description": "Represents a dataset containing raw intensities.\nHas support for sampling specific channels, normalizing\nwith provided scale and shifting, or reading in normalization\nbounds from OMERO metadata.", "type": "object", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "raw", "default": "raw", "title": "Dataset Type", "type": "string" }, "channels": { "anyOf": [ { "items": { "type": "integer" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Channels" }, "ome_norm": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "string" }, { "type": "null" } ], "default": null, "title": "Ome Norm" }, "scale_shift": { "anyOf": [ { "maxItems": 2, "minItems": 2, "prefixItems": [ { "type": "number" }, { "type": "number" } ], "type": "array" }, { "type": "null" } ], "default": null, "title": "Scale Shift" }, "stack": { "anyOf": [ { "$ref": "#/$defs/Dataset" }, { "type": "null" } ], "default": null } }, "$defs": { "Dataset": { "additionalProperties": false, "description": "A Dataset base class that defines the common attributes and methods\nfor all dataset types.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" } }, "required": [ "store" ], "title": "Dataset", "type": "object" } }, "additionalProperties": false, "required": [ "store" ] }
- Config:
extra: str = forbid
- Fields:
-
field channels:
list[int] |None= None#
-
field dataset_type:
Literal['raw'] = 'raw'#
-
field ome_norm:
Path|str|None= None#
-
field scale_shift:
tuple[float,float] |None= None#
- array(mode='r')#
- property attrs#
- property bounds: list[tuple[float, float]] | None#
Blockwise Tasks#
- volara.blockwise.get_task(task_type)#
Get the task class for a given task type. This is useful for dynamically fetching tasks from the base volara.blockwise module, despite some tasks potentially being defined in other modules as plugins via entry points. This allows volara to serialize and execute external tasks in isolated environments such as on cluster workers.
- Return type:
- pydantic model volara.blockwise.BlockwiseTask#
Show JSON schema
{ "title": "BlockwiseTask", "type": "object", "properties": { "roi": { "anyOf": [ { "maxItems": 2, "minItems": 2, "prefixItems": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "items": { "type": "integer" }, "minItems": 1, "type": "array" } ], "type": "array" }, { "type": "null" } ], "default": null, "title": "Roi" }, "num_workers": { "default": 1, "title": "Num Workers", "type": "integer" }, "num_cache_workers": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Num Cache Workers" }, "worker_config": { "anyOf": [ { "$ref": "#/$defs/Worker" }, { "type": "null" } ], "default": null }, "fit": { "title": "Fit", "type": "string" }, "read_write_conflict": { "title": "Read Write Conflict", "type": "boolean" } }, "$defs": { "Worker": { "additionalProperties": false, "properties": { "queue": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Queue" }, "num_gpus": { "default": 0, "title": "Num Gpus", "type": "integer" }, "num_cpus": { "default": 1, "title": "Num Cpus", "type": "integer" } }, "title": "Worker", "type": "object" } }, "additionalProperties": false, "required": [ "fit", "read_write_conflict" ] }
- Config:
extra: str = forbid
- Fields:
fit (str)num_cache_workers (int | None)num_workers (int)read_write_conflict (bool)roi (tuple[volara.utils.PydanticCoordinate, volara.utils.PydanticCoordinate] | None)worker_config (volara.workers.Worker | None)
-
field fit:
str[Required]# The strategy to use for blocks that overhang your total write roi.
-
field num_cache_workers:
int|None= None# The number of threads running the process_block function per worker. This allows you to start e.g. 4 gpu workers, each with 1 copy of the gpu loaded and running 8 threads to read, pre/postprocess, and write your data; maximizing your gpu utilization.
-
field num_workers:
int= 1# The number of workers that will be started to process blocks in parallel
-
field read_write_conflict:
bool[Required]# Whether blocks have read/write dependencies on neighborhing blocks requiring a specific ordering to the block processing to compute a seamless result.
-
field roi:
tuple[PydanticCoordinate,PydanticCoordinate] |None= None# An optional roi defining the total region to output.
-
field worker_config:
Worker|None= None# The configuration for each worker you start. This allows you to specify arguments for running workers on various platforms such as slurm/lsf clusters or AWS EC2.
- check_block_func()#
A function to check whether a block has been completed.
- drop(drop_outputs=False)#
A helper function to drop any artifacts produced by a task and return to a state identical to before having executed the task.
- Return type:
None
- abstract drop_artifacts()#
A helper function to reset anything produced by a task to a clean state equivalent to not having run the task at all
- init()#
Any one time initializations that need to be made before starting a task such as creating dbs and zarrs.
- init_block_array()#
Build the block done zarr for tracking completed blocks.
- mark_block_done_func()#
A helper function to mark a block as completed so that it can be skipped if we have to pause and resume processing later.
- model_post_init(context, /)#
This function is meant to behave like a BaseModel method to initialise private attributes.
It takes context as an argument since that’s what pydantic-core passes when calling it.
- Parameters:
self (
BaseModel) – The BaseModel instance.context (
Any) – The context.
- Return type:
None
- abstract process_block_func()#
A constructor for a function that will take a single block as input and process it.
- process_blocks()#
Start our workers and run through every block until a task is complete.
- process_roi(roi, context=None)#
A helper function to process a given roi without needing to start a whole blockwise job.
- run_blockwise(upstream_tasks=None, multiprocessing=True)#
Execute this task blockwise.
- task(upstream_tasks=None, multiprocessing=True)#
Builds a daisy.Task that puts together everything necessary to run a task blockwise.
- Return type:
Task
- worker_func()#
The function defining how workers are started.
- property block_ds: Path#
The dataset that will be used to track which blocks have been successfully completed.
- property block_write_roi: Roi#
The write roi of a block with zero offset
- property config_file: Path#
The config file that will be used to serialize this task for logging purposes.
- abstract property context_size: Coordinate | tuple[Coordinate, Coordinate]#
The amount of context needed to process each block for a task.
- property meta_dir: Path#
The path to the meta directory where we will store log files and a block done cache for resuming work if processing is interrupted.
- abstract property task_name: str#
A unique identifier for a task. This allows us to store log files in an unambiguous location as well as storing a block_done dataset that allows us to cache completed blocks and resume processing at a later time.
- abstract property write_roi: Roi#
The total roi of any data output by a task.
- abstract property write_size: Coordinate#
The write size of each block processed as part of a task.
- pydantic model volara.blockwise.AffAgglom#
A blockwise task that computes edges between supervoxels sharing affinity edges and stores statistics such as the mean affinity value between the supervoxels.
Show JSON schema
{ "title": "AffAgglom", "description": "A blockwise task that computes edges between supervoxels\nsharing affinity edges and stores statistics such as the\nmean affinity value between the supervoxels.", "type": "object", "properties": { "roi": { "anyOf": [ { "maxItems": 2, "minItems": 2, "prefixItems": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "items": { "type": "integer" }, "minItems": 1, "type": "array" } ], "type": "array" }, { "type": "null" } ], "default": null, "title": "Roi" }, "num_workers": { "default": 1, "title": "Num Workers", "type": "integer" }, "num_cache_workers": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Num Cache Workers" }, "worker_config": { "anyOf": [ { "$ref": "#/$defs/Worker" }, { "type": "null" } ], "default": null }, "fit": { "const": "shrink", "default": "shrink", "title": "Fit", "type": "string" }, "read_write_conflict": { "const": false, "default": false, "title": "Read Write Conflict", "type": "boolean" }, "task_type": { "const": "aff-agglom", "default": "aff-agglom", "title": "Task Type", "type": "string" }, "db": { "discriminator": { "mapping": { "postgresql": "#/$defs/PostgreSQL", "sqlite": "#/$defs/SQLite" }, "propertyName": "db_type" }, "oneOf": [ { "$ref": "#/$defs/PostgreSQL" }, { "$ref": "#/$defs/SQLite" } ], "title": "Db" }, "affs_data": { "$ref": "#/$defs/Affs" }, "frags_data": { "$ref": "#/$defs/Labels" }, "block_size": { "items": { "type": "integer" }, "minItems": 1, "title": "Block Size", "type": "array" }, "context": { "items": { "type": "integer" }, "minItems": 1, "title": "Context", "type": "array" }, "scores": { "additionalProperties": { "items": { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, "type": "array" }, "title": "Scores", "type": "object" } }, "$defs": { "Affs": { "additionalProperties": false, "description": "Represents a dataset containing affinities.\nRequires the inclusion of the neighborhood for these\naffinities.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "affs", "default": "affs", "title": "Dataset Type", "type": "string" }, "neighborhood": { "anyOf": [ { "items": { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Neighborhood" } }, "required": [ "store" ], "title": "Affs", "type": "object" }, "Labels": { "additionalProperties": false, "description": "Represents an integer label dataset.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "labels", "default": "labels", "title": "Dataset Type", "type": "string" } }, "required": [ "store" ], "title": "Labels", "type": "object" }, "PostgreSQL": { "additionalProperties": false, "description": "A PostgreSQL database for storing and retrieving graph data.", "properties": { "node_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Node Attrs" }, "edge_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Edge Attrs" }, "ndim": { "default": 3, "title": "Ndim", "type": "integer" }, "db_type": { "const": "postgresql", "default": "postgresql", "title": "Db Type", "type": "string" }, "host": { "default": "localhost", "title": "Host", "type": "string" }, "name": { "default": "volara", "title": "Name", "type": "string" }, "user": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "User" }, "password": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Password" } }, "title": "PostgreSQL", "type": "object" }, "SQLite": { "additionalProperties": false, "description": "An SQLite database for storing and retrieving graph data.", "properties": { "node_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Node Attrs" }, "edge_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Edge Attrs" }, "ndim": { "default": 3, "title": "Ndim", "type": "integer" }, "db_type": { "const": "sqlite", "default": "sqlite", "title": "Db Type", "type": "string" }, "path": { "format": "path", "title": "Path", "type": "string" } }, "required": [ "path" ], "title": "SQLite", "type": "object" }, "Worker": { "additionalProperties": false, "properties": { "queue": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Queue" }, "num_gpus": { "default": 0, "title": "Num Gpus", "type": "integer" }, "num_cpus": { "default": 1, "title": "Num Cpus", "type": "integer" } }, "title": "Worker", "type": "object" } }, "additionalProperties": false, "required": [ "db", "affs_data", "frags_data", "block_size", "context", "scores" ] }
- Config:
extra: str = forbid
- Fields:
affs_data (volara.datasets.Affs)block_size (volara.utils.PydanticCoordinate)context (volara.utils.PydanticCoordinate)db (Annotated[volara.dbs.PostgreSQL | volara.dbs.SQLite, FieldInfo(annotation=NoneType, required=True, discriminator='db_type')])fit (Literal['shrink'])frags_data (volara.datasets.Labels)read_write_conflict (Literal[False])scores (dict[str, list[volara.utils.PydanticCoordinate]])task_type (Literal['aff-agglom'])
-
field block_size:
PydanticCoordinate[Required]# The blocksize within which to compute supervoxel affinity statistics
-
field context:
PydanticCoordinate[Required]# The amount of context to use when computing supervoxel affinity statistics
-
field db:
Annotated[PostgreSQL|SQLite] [Required]# The database containing nodes associated with the fragment supervoxels to which we will add the supervoxel edge affinity statistics
-
field fit:
Literal['shrink'] = 'shrink'# The boundary behavior for our daisy task.
-
field frags_data:
Labels[Required]# The labels array containing the supervoxels for computing affinity statistics
-
field read_write_conflict:
Literal[False] = False# We don’t have read write conflicts in this task and can compute every block independently in an arbitrary order.
-
field scores:
dict[str,list[PydanticCoordinate]] [Required]# A dictionary of score names and their respective neighborhoods. This allows us to compute the affinity statistics in subgroups of the neighborhood. For example if you wanted to compute the mean affinity between fragments in x and y separately from z, you would provide a dictionary like so:
- scores = {
“xy”: [(1, 0, 0), (0, 1, 0)], “z”: [(0, 0, 1)]
}
-
field task_type:
Literal['aff-agglom'] = 'aff-agglom'#
- agglomerate(affs, frags, rag)#
- agglomerate_in_block(block, affs, frags, rag_provider)#
- drop_artifacts()#
A helper function to reset anything produced by a task to a clean state equivalent to not having run the task at all
- init()#
Any one time initializations that need to be made before starting a task such as creating dbs and zarrs.
- Return type:
None
- model_post_init(context, /)#
This function is meant to behave like a BaseModel method to initialise private attributes.
It takes context as an argument since that’s what pydantic-core passes when calling it.
- Parameters:
self (
BaseModel) – The BaseModel instance.context (
Any) – The context.
- Return type:
None
- process_block_func()#
A constructor for a function that will take a single block as input and process it.
- Return type:
Generator[Callable,None,None]
- property context_size: Coordinate#
The amount of context needed to process each block for a task.
- property task_name: str#
A unique identifier for a task. This allows us to store log files in an unambiguous location as well as storing a block_done dataset that allows us to cache completed blocks and resume processing at a later time.
- property write_roi: Roi#
The total roi of any data output by a task.
- property write_size: Coordinate#
The write size of each block processed as part of a task.
- pydantic model volara.blockwise.ApplyShift#
Show JSON schema
{ "title": "ApplyShift", "type": "object", "properties": { "roi": { "anyOf": [ { "maxItems": 2, "minItems": 2, "prefixItems": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "items": { "type": "integer" }, "minItems": 1, "type": "array" } ], "type": "array" }, { "type": "null" } ], "default": null, "title": "Roi" }, "num_workers": { "default": 1, "title": "Num Workers", "type": "integer" }, "num_cache_workers": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Num Cache Workers" }, "worker_config": { "anyOf": [ { "$ref": "#/$defs/Worker" }, { "type": "null" } ], "default": null }, "fit": { "const": "overhang", "default": "overhang", "title": "Fit", "type": "string" }, "read_write_conflict": { "const": false, "default": false, "title": "Read Write Conflict", "type": "boolean" }, "task_type": { "const": "apply_shifts", "default": "apply_shifts", "title": "Task Type", "type": "string" }, "intensities": { "$ref": "#/$defs/Raw" }, "shifts": { "$ref": "#/$defs/Raw" }, "aligned": { "$ref": "#/$defs/Raw" }, "interp_shifts": { "anyOf": [ { "$ref": "#/$defs/Raw" }, { "type": "null" } ], "default": null }, "shift_threshold": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Shift Threshold" } }, "$defs": { "Dataset": { "additionalProperties": false, "description": "A Dataset base class that defines the common attributes and methods\nfor all dataset types.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" } }, "required": [ "store" ], "title": "Dataset", "type": "object" }, "Raw": { "additionalProperties": false, "description": "Represents a dataset containing raw intensities.\nHas support for sampling specific channels, normalizing\nwith provided scale and shifting, or reading in normalization\nbounds from OMERO metadata.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "raw", "default": "raw", "title": "Dataset Type", "type": "string" }, "channels": { "anyOf": [ { "items": { "type": "integer" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Channels" }, "ome_norm": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "string" }, { "type": "null" } ], "default": null, "title": "Ome Norm" }, "scale_shift": { "anyOf": [ { "maxItems": 2, "minItems": 2, "prefixItems": [ { "type": "number" }, { "type": "number" } ], "type": "array" }, { "type": "null" } ], "default": null, "title": "Scale Shift" }, "stack": { "anyOf": [ { "$ref": "#/$defs/Dataset" }, { "type": "null" } ], "default": null } }, "required": [ "store" ], "title": "Raw", "type": "object" }, "Worker": { "additionalProperties": false, "properties": { "queue": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Queue" }, "num_gpus": { "default": 0, "title": "Num Gpus", "type": "integer" }, "num_cpus": { "default": 1, "title": "Num Cpus", "type": "integer" } }, "title": "Worker", "type": "object" } }, "additionalProperties": false, "required": [ "intensities", "shifts", "aligned" ] }
- Config:
extra: str = forbid
- Fields:
aligned (volara.datasets.Raw)fit (Literal['overhang'])intensities (volara.datasets.Raw)interp_shifts (volara.datasets.Raw | None)read_write_conflict (Literal[False])shift_threshold (float | None)shifts (volara.datasets.Raw)task_type (Literal['apply_shifts'])
-
field fit:
Literal['overhang'] = 'overhang'# The strategy to use for blocks that overhang your total write roi.
-
field read_write_conflict:
Literal[False] = False# Whether blocks have read/write dependencies on neighborhing blocks requiring a specific ordering to the block processing to compute a seamless result.
-
field shift_threshold:
float|None= None#
-
field task_type:
Literal['apply_shifts'] = 'apply_shifts'#
- static apply_shift(intensities, shifts, voxel_write_roi, voxel_size, shift_threshold=None)#
- Return type:
tuple[ndarray,ndarray]
- drop_artifacts()#
A helper function to reset anything produced by a task to a clean state equivalent to not having run the task at all
- init()#
Any one time initializations that need to be made before starting a task such as creating dbs and zarrs.
- init_out_array()#
- model_post_init(context, /)#
This function is meant to behave like a BaseModel method to initialise private attributes.
It takes context as an argument since that’s what pydantic-core passes when calling it.
- Parameters:
self (
BaseModel) – The BaseModel instance.context (
Any) – The context.
- Return type:
None
- process_block_func()#
A constructor for a function that will take a single block as input and process it.
- property block_size: Coordinate#
- property context#
- property context_size#
The amount of context needed to process each block for a task.
- property task_name: str#
A unique identifier for a task. This allows us to store log files in an unambiguous location as well as storing a block_done dataset that allows us to cache completed blocks and resume processing at a later time.
- property voxel_size: Coordinate#
- property write_roi: Roi#
The total roi of any data output by a task.
- property write_size: Coordinate#
The write size of each block processed as part of a task.
- pydantic model volara.blockwise.Argmax#
A blockwise task that performs an argmax operation on a given set of probabilities and writes the result to a semantic segmentation dataset.
Show JSON schema
{ "title": "Argmax", "description": "A blockwise task that performs an argmax operation on a given set of\nprobabilities and writes the result to a semantic segmentation dataset.", "type": "object", "properties": { "roi": { "anyOf": [ { "maxItems": 2, "minItems": 2, "prefixItems": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "items": { "type": "integer" }, "minItems": 1, "type": "array" } ], "type": "array" }, { "type": "null" } ], "default": null, "title": "Roi" }, "num_workers": { "default": 1, "title": "Num Workers", "type": "integer" }, "num_cache_workers": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Num Cache Workers" }, "worker_config": { "anyOf": [ { "$ref": "#/$defs/Worker" }, { "type": "null" } ], "default": null }, "fit": { "const": "shrink", "default": "shrink", "title": "Fit", "type": "string" }, "read_write_conflict": { "const": false, "default": false, "title": "Read Write Conflict", "type": "boolean" }, "task_type": { "const": "argmax", "default": "argmax", "title": "Task Type", "type": "string" }, "probs_data": { "$ref": "#/$defs/Raw" }, "sem_data": { "$ref": "#/$defs/Labels" }, "combine_classes": { "anyOf": [ { "items": { "items": { "type": "integer" }, "type": "array" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Combine Classes" }, "block_size": { "items": { "type": "integer" }, "minItems": 1, "title": "Block Size", "type": "array" } }, "$defs": { "Dataset": { "additionalProperties": false, "description": "A Dataset base class that defines the common attributes and methods\nfor all dataset types.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" } }, "required": [ "store" ], "title": "Dataset", "type": "object" }, "Labels": { "additionalProperties": false, "description": "Represents an integer label dataset.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "labels", "default": "labels", "title": "Dataset Type", "type": "string" } }, "required": [ "store" ], "title": "Labels", "type": "object" }, "Raw": { "additionalProperties": false, "description": "Represents a dataset containing raw intensities.\nHas support for sampling specific channels, normalizing\nwith provided scale and shifting, or reading in normalization\nbounds from OMERO metadata.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "raw", "default": "raw", "title": "Dataset Type", "type": "string" }, "channels": { "anyOf": [ { "items": { "type": "integer" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Channels" }, "ome_norm": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "string" }, { "type": "null" } ], "default": null, "title": "Ome Norm" }, "scale_shift": { "anyOf": [ { "maxItems": 2, "minItems": 2, "prefixItems": [ { "type": "number" }, { "type": "number" } ], "type": "array" }, { "type": "null" } ], "default": null, "title": "Scale Shift" }, "stack": { "anyOf": [ { "$ref": "#/$defs/Dataset" }, { "type": "null" } ], "default": null } }, "required": [ "store" ], "title": "Raw", "type": "object" }, "Worker": { "additionalProperties": false, "properties": { "queue": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Queue" }, "num_gpus": { "default": 0, "title": "Num Gpus", "type": "integer" }, "num_cpus": { "default": 1, "title": "Num Cpus", "type": "integer" } }, "title": "Worker", "type": "object" } }, "additionalProperties": false, "required": [ "probs_data", "sem_data", "block_size" ] }
- Config:
extra: str = forbid
- Fields:
block_size (volara.utils.PydanticCoordinate)combine_classes (list[list[int]] | None)fit (Literal['shrink'])probs_data (volara.datasets.Raw)read_write_conflict (Literal[False])sem_data (volara.datasets.Labels)task_type (Literal['argmax'])
-
field block_size:
PydanticCoordinate[Required]# The block size with which to chunk our argmax task.
-
field combine_classes:
list[list[int]] |None= None# A list of lists containing the ids to combine. All channels in combine_classes[i] will be summed into a new channel i before computing the argmax.
-
field fit:
Literal['shrink'] = 'shrink'# The strategy to use for blocks that overhang your total write roi.
-
field probs_data:
Raw[Required]# The dataset containing raw probabilities for which you want to compute the argmax.
-
field read_write_conflict:
Literal[False] = False# Whether blocks have read/write dependencies on neighborhing blocks requiring a specific ordering to the block processing to compute a seamless result.
-
field task_type:
Literal['argmax'] = 'argmax'#
- argmax_block(block, probabilities, semantic)#
- drop_artifacts()#
A helper function to reset anything produced by a task to a clean state equivalent to not having run the task at all
- init()#
Any one time initializations that need to be made before starting a task such as creating dbs and zarrs.
- init_out_array()#
- model_post_init(context, /)#
This function is meant to behave like a BaseModel method to initialise private attributes.
It takes context as an argument since that’s what pydantic-core passes when calling it.
- Parameters:
self (
BaseModel) – The BaseModel instance.context (
Any) – The context.
- Return type:
None
- process_block_func()#
A constructor for a function that will take a single block as input and process it.
- property context_size: Coordinate#
The amount of context needed to process each block for a task.
- property task_name: str#
A unique identifier for a task. This allows us to store log files in an unambiguous location as well as storing a block_done dataset that allows us to cache completed blocks and resume processing at a later time.
- property voxel_size: Coordinate#
- property write_roi: Roi#
The total roi of any data output by a task.
- property write_size: Coordinate#
The write size of each block processed as part of a task.
- pydantic model volara.blockwise.ComputeShift#
Show JSON schema
{ "title": "ComputeShift", "type": "object", "properties": { "roi": { "anyOf": [ { "maxItems": 2, "minItems": 2, "prefixItems": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "items": { "type": "integer" }, "minItems": 1, "type": "array" } ], "type": "array" }, { "type": "null" } ], "default": null, "title": "Roi" }, "num_workers": { "default": 1, "title": "Num Workers", "type": "integer" }, "num_cache_workers": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Num Cache Workers" }, "worker_config": { "anyOf": [ { "$ref": "#/$defs/Worker" }, { "type": "null" } ], "default": null }, "fit": { "const": "overhang", "default": "overhang", "title": "Fit", "type": "string" }, "read_write_conflict": { "const": false, "default": false, "title": "Read Write Conflict", "type": "boolean" }, "task_type": { "const": "compute_shift", "default": "compute_shift", "title": "Task Type", "type": "string" }, "intensities": { "$ref": "#/$defs/Raw" }, "shifts": { "$ref": "#/$defs/Raw" }, "block_size": { "items": { "type": "integer" }, "minItems": 1, "title": "Block Size", "type": "array" }, "context": { "items": { "type": "integer" }, "minItems": 1, "title": "Context", "type": "array" }, "mask": { "anyOf": [ { "$ref": "#/$defs/Labels" }, { "type": "null" } ], "default": null }, "overlap_ratio": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Overlap Ratio" }, "target": { "anyOf": [ { "$ref": "#/$defs/Raw" }, { "type": "null" } ], "default": null } }, "$defs": { "Dataset": { "additionalProperties": false, "description": "A Dataset base class that defines the common attributes and methods\nfor all dataset types.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" } }, "required": [ "store" ], "title": "Dataset", "type": "object" }, "Labels": { "additionalProperties": false, "description": "Represents an integer label dataset.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "labels", "default": "labels", "title": "Dataset Type", "type": "string" } }, "required": [ "store" ], "title": "Labels", "type": "object" }, "Raw": { "additionalProperties": false, "description": "Represents a dataset containing raw intensities.\nHas support for sampling specific channels, normalizing\nwith provided scale and shifting, or reading in normalization\nbounds from OMERO metadata.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "raw", "default": "raw", "title": "Dataset Type", "type": "string" }, "channels": { "anyOf": [ { "items": { "type": "integer" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Channels" }, "ome_norm": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "string" }, { "type": "null" } ], "default": null, "title": "Ome Norm" }, "scale_shift": { "anyOf": [ { "maxItems": 2, "minItems": 2, "prefixItems": [ { "type": "number" }, { "type": "number" } ], "type": "array" }, { "type": "null" } ], "default": null, "title": "Scale Shift" }, "stack": { "anyOf": [ { "$ref": "#/$defs/Dataset" }, { "type": "null" } ], "default": null } }, "required": [ "store" ], "title": "Raw", "type": "object" }, "Worker": { "additionalProperties": false, "properties": { "queue": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Queue" }, "num_gpus": { "default": 0, "title": "Num Gpus", "type": "integer" }, "num_cpus": { "default": 1, "title": "Num Cpus", "type": "integer" } }, "title": "Worker", "type": "object" } }, "additionalProperties": false, "required": [ "intensities", "shifts", "block_size", "context" ] }
- Config:
extra: str = forbid
- Fields:
block_size (volara.utils.PydanticCoordinate)context (volara.utils.PydanticCoordinate)fit (Literal['overhang'])intensities (volara.datasets.Raw)mask (volara.datasets.Labels | None)overlap_ratio (float | None)read_write_conflict (Literal[False])shifts (volara.datasets.Raw)target (volara.datasets.Raw | None)task_type (Literal['compute_shift'])
-
field block_size:
PydanticCoordinate[Required]#
-
field context:
PydanticCoordinate[Required]#
-
field fit:
Literal['overhang'] = 'overhang'# The strategy to use for blocks that overhang your total write roi.
-
field overlap_ratio:
float|None= None#
-
field read_write_conflict:
Literal[False] = False# Whether blocks have read/write dependencies on neighborhing blocks requiring a specific ordering to the block processing to compute a seamless result.
-
field task_type:
Literal['compute_shift'] = 'compute_shift'#
- static compute_shift(array, target_data, voxel_size, mask=None, overlap_ratio=None)#
- Return type:
ndarray
- drop_artifacts()#
A helper function to reset anything produced by a task to a clean state equivalent to not having run the task at all
- init()#
Any one time initializations that need to be made before starting a task such as creating dbs and zarrs.
- init_out_array()#
- model_post_init(context, /)#
This function is meant to behave like a BaseModel method to initialise private attributes.
It takes context as an argument since that’s what pydantic-core passes when calling it.
- Parameters:
self (
BaseModel) – The BaseModel instance.context (
Any) – The context.
- Return type:
None
- process_block_func()#
A constructor for a function that will take a single block as input and process it.
- property context_size#
The amount of context needed to process each block for a task.
- property task_name: str#
A unique identifier for a task. This allows us to store log files in an unambiguous location as well as storing a block_done dataset that allows us to cache completed blocks and resume processing at a later time.
- property voxel_size: Coordinate#
- property write_roi: Roi#
The total roi of any data output by a task.
- property write_size: Coordinate#
The write size of each block processed as part of a task.
- pydantic model volara.blockwise.DistanceAgglom#
Distance based edge creation for fragment to segment agglomeration. In this context distance is defined as the distance between the embeddings of two fragments. Given some attributes saved or computable on the fragments we compute the distance between pairs of fragments and then save the edges with their costs. Because the distance is computed between embeddings, there is no spatial limit to the distance between fragments. This means that without a spatial constraint we would need to compute the distance between all pairs of fragments. To avoid this we can set a connection radius that will limit the euclidean distance between connectable fragments.
Show JSON schema
{ "title": "DistanceAgglom", "description": "Distance based edge creation for fragment to segment agglomeration.\nIn this context distance is defined as the distance between the embeddings\nof two fragments. Given some attributes saved or computable on the fragments\nwe compute the distance between pairs of fragments and then save the edges\nwith their costs.\nBecause the distance is computed between embeddings, there is no spatial limit\nto the distance between fragments. This means that without a spatial constraint\nwe would need to compute the distance between all pairs of fragments. To avoid\nthis we can set a connection radius that will limit the euclidean distance between\nconnectable fragments.", "type": "object", "properties": { "roi": { "anyOf": [ { "maxItems": 2, "minItems": 2, "prefixItems": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "items": { "type": "integer" }, "minItems": 1, "type": "array" } ], "type": "array" }, { "type": "null" } ], "default": null, "title": "Roi" }, "num_workers": { "default": 1, "title": "Num Workers", "type": "integer" }, "num_cache_workers": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Num Cache Workers" }, "worker_config": { "anyOf": [ { "$ref": "#/$defs/Worker" }, { "type": "null" } ], "default": null }, "fit": { "const": "shrink", "default": "shrink", "title": "Fit", "type": "string" }, "read_write_conflict": { "const": false, "default": false, "title": "Read Write Conflict", "type": "boolean" }, "task_type": { "const": "distance-agglom", "default": "distance-agglom", "title": "Task Type", "type": "string" }, "storage": { "anyOf": [ { "discriminator": { "mapping": { "postgresql": "#/$defs/PostgreSQL", "sqlite": "#/$defs/SQLite" }, "propertyName": "db_type" }, "oneOf": [ { "$ref": "#/$defs/PostgreSQL" }, { "$ref": "#/$defs/SQLite" } ] }, { "$ref": "#/$defs/LUT" } ], "title": "Storage" }, "frags_data": { "$ref": "#/$defs/Labels" }, "distance_keys": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Distance Keys" }, "background_intensities": { "anyOf": [ { "items": { "type": "number" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Background Intensities" }, "eps": { "default": 1e-08, "title": "Eps", "type": "number" }, "distance_threshold": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Distance Threshold" }, "distance_metric": { "default": "cosine", "enum": [ "euclidean", "cosine", "max" ], "title": "Distance Metric", "type": "string" }, "block_size": { "items": { "type": "integer" }, "minItems": 1, "title": "Block Size", "type": "array" }, "context": { "items": { "type": "integer" }, "minItems": 1, "title": "Context", "type": "array" } }, "$defs": { "LUT": { "additionalProperties": false, "description": "A class for defining look up tables", "properties": { "path": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "string" } ], "title": "Path" } }, "required": [ "path" ], "title": "LUT", "type": "object" }, "Labels": { "additionalProperties": false, "description": "Represents an integer label dataset.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "labels", "default": "labels", "title": "Dataset Type", "type": "string" } }, "required": [ "store" ], "title": "Labels", "type": "object" }, "PostgreSQL": { "additionalProperties": false, "description": "A PostgreSQL database for storing and retrieving graph data.", "properties": { "node_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Node Attrs" }, "edge_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Edge Attrs" }, "ndim": { "default": 3, "title": "Ndim", "type": "integer" }, "db_type": { "const": "postgresql", "default": "postgresql", "title": "Db Type", "type": "string" }, "host": { "default": "localhost", "title": "Host", "type": "string" }, "name": { "default": "volara", "title": "Name", "type": "string" }, "user": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "User" }, "password": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Password" } }, "title": "PostgreSQL", "type": "object" }, "SQLite": { "additionalProperties": false, "description": "An SQLite database for storing and retrieving graph data.", "properties": { "node_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Node Attrs" }, "edge_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Edge Attrs" }, "ndim": { "default": 3, "title": "Ndim", "type": "integer" }, "db_type": { "const": "sqlite", "default": "sqlite", "title": "Db Type", "type": "string" }, "path": { "format": "path", "title": "Path", "type": "string" } }, "required": [ "path" ], "title": "SQLite", "type": "object" }, "Worker": { "additionalProperties": false, "properties": { "queue": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Queue" }, "num_gpus": { "default": 0, "title": "Num Gpus", "type": "integer" }, "num_cpus": { "default": 1, "title": "Num Cpus", "type": "integer" } }, "title": "Worker", "type": "object" } }, "additionalProperties": false, "required": [ "storage", "frags_data", "block_size", "context" ] }
- Config:
extra: str = forbid
- Fields:
background_intensities (list[float] | None)block_size (volara.utils.PydanticCoordinate)context (volara.utils.PydanticCoordinate)distance_keys (list[str] | None)distance_metric (Literal['euclidean', 'cosine', 'max'])distance_threshold (float | None)eps (float)fit (Literal['shrink'])frags_data (volara.datasets.Labels)read_write_conflict (Literal[False])storage (Annotated[volara.dbs.PostgreSQL | volara.dbs.SQLite, FieldInfo(annotation=NoneType, required=True, discriminator='db_type')] | volara.lut.LUT)task_type (Literal['distance-agglom'])
-
field background_intensities:
list[float] |None= None#
-
field block_size:
PydanticCoordinate[Required]#
-
field context:
PydanticCoordinate[Required]#
-
field distance_keys:
list[str] |None= None#
-
field distance_metric:
Literal['euclidean','cosine','max'] = 'cosine'#
-
field distance_threshold:
float|None= None#
-
field eps:
float= 1e-08#
-
field fit:
Literal['shrink'] = 'shrink'# The strategy to use for blocks that overhang your total write roi.
-
field read_write_conflict:
Literal[False] = False# Whether blocks have read/write dependencies on neighborhing blocks requiring a specific ordering to the block processing to compute a seamless result.
-
field storage:
Union[Annotated[PostgreSQL|SQLite],LUT] [Required]# Where to store the edges and or the final Look Up Table. If a Database is provided, it is assumed that each fragment in the fragments dataset has a node in the graph already saved.
-
field task_type:
Literal['distance-agglom'] = 'distance-agglom'#
- agglomerate_in_block(block, frags, rag_provider)#
- drop_artifacts()#
A helper function to reset anything produced by a task to a clean state equivalent to not having run the task at all
- label_distances(labels, voxel_size, dist_threshold=0.0)#
- model_post_init(context, /)#
This function is meant to behave like a BaseModel method to initialise private attributes.
It takes context as an argument since that’s what pydantic-core passes when calling it.
- Parameters:
self (
BaseModel) – The BaseModel instance.context (
Any) – The context.
- Return type:
None
- process_block_func()#
A constructor for a function that will take a single block as input and process it.
- property context_size: Coordinate#
The amount of context needed to process each block for a task.
- property task_name: str#
A unique identifier for a task. This allows us to store log files in an unambiguous location as well as storing a block_done dataset that allows us to cache completed blocks and resume processing at a later time.
- property write_roi: Roi#
The total roi of any data output by a task.
- property write_size: Coordinate#
The write size of each block processed as part of a task.
- pydantic model volara.blockwise.ExtractFrags#
A task for extracting fragments from affinities. Internally it uses mutex watershed to agglomerate fragments.
Show JSON schema
{ "title": "ExtractFrags", "description": "A task for extracting fragments from affinities.\nInternally it uses mutex watershed to agglomerate fragments.", "type": "object", "properties": { "roi": { "anyOf": [ { "maxItems": 2, "minItems": 2, "prefixItems": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "items": { "type": "integer" }, "minItems": 1, "type": "array" } ], "type": "array" }, { "type": "null" } ], "default": null, "title": "Roi" }, "num_workers": { "default": 1, "title": "Num Workers", "type": "integer" }, "num_cache_workers": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Num Cache Workers" }, "worker_config": { "anyOf": [ { "$ref": "#/$defs/Worker" }, { "type": "null" } ], "default": null }, "fit": { "const": "shrink", "default": "shrink", "title": "Fit", "type": "string" }, "read_write_conflict": { "const": false, "default": false, "title": "Read Write Conflict", "type": "boolean" }, "task_type": { "const": "extract-frags", "default": "extract-frags", "title": "Task Type", "type": "string" }, "db": { "discriminator": { "mapping": { "postgresql": "#/$defs/PostgreSQL", "sqlite": "#/$defs/SQLite" }, "propertyName": "db_type" }, "oneOf": [ { "$ref": "#/$defs/PostgreSQL" }, { "$ref": "#/$defs/SQLite" } ], "title": "Db" }, "affs_data": { "$ref": "#/$defs/Affs" }, "frags_data": { "$ref": "#/$defs/Labels" }, "mask_data": { "anyOf": [ { "$ref": "#/$defs/Raw" }, { "type": "null" } ], "default": null }, "block_size": { "items": { "type": "integer" }, "minItems": 1, "title": "Block Size", "type": "array" }, "context": { "items": { "type": "integer" }, "minItems": 1, "title": "Context", "type": "array" }, "bias": { "items": { "type": "number" }, "title": "Bias", "type": "array" }, "strides": { "anyOf": [ { "items": { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Strides" }, "sigma": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Sigma" }, "noise_eps": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Noise Eps" }, "filter_fragments": { "default": 0.0, "title": "Filter Fragments", "type": "number" }, "remove_debris": { "default": 0, "title": "Remove Debris", "type": "integer" }, "randomized_strides": { "default": false, "title": "Randomized Strides", "type": "boolean" } }, "$defs": { "Affs": { "additionalProperties": false, "description": "Represents a dataset containing affinities.\nRequires the inclusion of the neighborhood for these\naffinities.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "affs", "default": "affs", "title": "Dataset Type", "type": "string" }, "neighborhood": { "anyOf": [ { "items": { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Neighborhood" } }, "required": [ "store" ], "title": "Affs", "type": "object" }, "Dataset": { "additionalProperties": false, "description": "A Dataset base class that defines the common attributes and methods\nfor all dataset types.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" } }, "required": [ "store" ], "title": "Dataset", "type": "object" }, "Labels": { "additionalProperties": false, "description": "Represents an integer label dataset.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "labels", "default": "labels", "title": "Dataset Type", "type": "string" } }, "required": [ "store" ], "title": "Labels", "type": "object" }, "PostgreSQL": { "additionalProperties": false, "description": "A PostgreSQL database for storing and retrieving graph data.", "properties": { "node_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Node Attrs" }, "edge_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Edge Attrs" }, "ndim": { "default": 3, "title": "Ndim", "type": "integer" }, "db_type": { "const": "postgresql", "default": "postgresql", "title": "Db Type", "type": "string" }, "host": { "default": "localhost", "title": "Host", "type": "string" }, "name": { "default": "volara", "title": "Name", "type": "string" }, "user": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "User" }, "password": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Password" } }, "title": "PostgreSQL", "type": "object" }, "Raw": { "additionalProperties": false, "description": "Represents a dataset containing raw intensities.\nHas support for sampling specific channels, normalizing\nwith provided scale and shifting, or reading in normalization\nbounds from OMERO metadata.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "raw", "default": "raw", "title": "Dataset Type", "type": "string" }, "channels": { "anyOf": [ { "items": { "type": "integer" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Channels" }, "ome_norm": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "string" }, { "type": "null" } ], "default": null, "title": "Ome Norm" }, "scale_shift": { "anyOf": [ { "maxItems": 2, "minItems": 2, "prefixItems": [ { "type": "number" }, { "type": "number" } ], "type": "array" }, { "type": "null" } ], "default": null, "title": "Scale Shift" }, "stack": { "anyOf": [ { "$ref": "#/$defs/Dataset" }, { "type": "null" } ], "default": null } }, "required": [ "store" ], "title": "Raw", "type": "object" }, "SQLite": { "additionalProperties": false, "description": "An SQLite database for storing and retrieving graph data.", "properties": { "node_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Node Attrs" }, "edge_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Edge Attrs" }, "ndim": { "default": 3, "title": "Ndim", "type": "integer" }, "db_type": { "const": "sqlite", "default": "sqlite", "title": "Db Type", "type": "string" }, "path": { "format": "path", "title": "Path", "type": "string" } }, "required": [ "path" ], "title": "SQLite", "type": "object" }, "Worker": { "additionalProperties": false, "properties": { "queue": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Queue" }, "num_gpus": { "default": 0, "title": "Num Gpus", "type": "integer" }, "num_cpus": { "default": 1, "title": "Num Cpus", "type": "integer" } }, "title": "Worker", "type": "object" } }, "additionalProperties": false, "required": [ "db", "affs_data", "frags_data", "block_size", "context", "bias" ] }
- Config:
extra: str = forbid
- Fields:
affs_data (volara.datasets.Affs)bias (list[float])block_size (volara.utils.PydanticCoordinate)context (volara.utils.PydanticCoordinate)db (Annotated[volara.dbs.PostgreSQL | volara.dbs.SQLite, FieldInfo(annotation=NoneType, required=True, discriminator='db_type')])filter_fragments (float)fit (Literal['shrink'])frags_data (volara.datasets.Labels)mask_data (volara.datasets.Raw | None)noise_eps (float | None)randomized_strides (bool)read_write_conflict (Literal[False])remove_debris (int)sigma (volara.utils.PydanticCoordinate | None)strides (list[volara.utils.PydanticCoordinate] | None)task_type (Literal['extract-frags'])
-
field bias:
list[float] [Required]# The merge/split bias for the affinities. This should be a vector of length equal to the size of the neighborhood with one bias per offset. This allows you to have a merge preferring bias for short range affinites and a split preferring bias for long range affinities.
Example: Assuming you trained affs [(0, 1), (1, 0), (0, 4), (4, 0)] for a 2D dataset, you can set the bias to [-0.2, -0.2, -0.8, -0.8]. This will bias you towards merging on short range affinities and splitting on long range affinities which has been shown to work well.
-
field block_size:
PydanticCoordinate[Required]#
-
field context:
PydanticCoordinate[Required]#
-
field db:
Annotated[PostgreSQL|SQLite] [Required]# The database into which we will store node centers along with statistics such as fragment size, mean intensity, etc.
-
field filter_fragments:
float= 0.0# The minimum average affinity value for a fragment to be considered valid. If the average affinity value is below this threshold the fragment will be removed.
-
field fit:
Literal['shrink'] = 'shrink'# The strategy to use for blocks that overhang your total write roi.
-
field noise_eps:
float|None= None# The amplitude of the random noise to add to the affinities before watershed. This also helps avoid streak like fragment artifacts from processing affinities in a fifo order.
-
field randomized_strides:
bool= False# If using strides, you may want to switch from a grided stride to a random probability of filtering out an affinity. This can help avoid grid artifacts in the fragments.
-
field read_write_conflict:
Literal[False] = False# Whether blocks have read/write dependencies on neighborhing blocks requiring a specific ordering to the block processing to compute a seamless result.
-
field remove_debris:
int= 0# The minimum size of a fragment to be considered valid. If the fragment is smaller than this value it will be removed.
-
field sigma:
PydanticCoordinate|None= None# The amplitude of the smoothing kernel to apply to the affinities before watershed. This can help agglomerate fragments from the inside out to avoid a small merge error causing a large fragment to split in half.
-
field strides:
list[PydanticCoordinate] |None= None# The strides to use for each affinity offset in the mutex watershed algorithm. If you have long range affinities it can be heplful to ignore some percentage of them to avoid excessive splits, so you may want to use only every other voxel in the z direction for example.
Example: Assuming you trained affs [(0, 1), (1, 0), (0, 4), (4, 0)] for a 2D dataset, you can set the strides to [(1, 1), (1, 1), (2, 2), (2, 2)]. This will result in only 1 in every 4 long range affinities being used in the mutex watershed algorithm resulting in fewer splits (assuming you biased long range affinities towards splitting).
-
field task_type:
Literal['extract-frags'] = 'extract-frags'#
- compute_fragments(affs_data)#
- drop_artifacts()#
A helper function to reset anything produced by a task to a clean state equivalent to not having run the task at all
- filter_avg_fragments(affs, fragments_data, filter_value)#
- get_fragments(affs_data)#
- init()#
Any one time initializations that need to be made before starting a task such as creating dbs and zarrs.
- init_out_array()#
- model_post_init(context, /)#
This function is meant to behave like a BaseModel method to initialise private attributes.
It takes context as an argument since that’s what pydantic-core passes when calling it.
- Parameters:
self (
BaseModel) – The BaseModel instance.context (
Any) – The context.
- Return type:
None
- process_block_func()#
A constructor for a function that will take a single block as input and process it.
- watershed_in_block(block, affs, frags, rag_provider, mask=None)#
- property context_size: Coordinate#
The amount of context needed to process each block for a task.
- property neighborhood#
- property num_voxels_in_block: int#
- property task_name: str#
A unique identifier for a task. This allows us to store log files in an unambiguous location as well as storing a block_done dataset that allows us to cache completed blocks and resume processing at a later time.
- property voxel_size: Coordinate#
- property write_roi: Roi#
The total roi of any data output by a task.
- property write_size: Coordinate#
The write size of each block processed as part of a task.
- pydantic model volara.blockwise.GraphMWS#
Graph based execution of the MWS algorithm. Currently only supports executing in memory. The full graph for the given ROI is read into memory and then we run the mutex watershed algorithm on the full graph to get a globally optimal look up table.
Show JSON schema
{ "title": "GraphMWS", "description": "Graph based execution of the MWS algorithm.\nCurrently only supports executing in memory. The full graph for the given ROI\nis read into memory and then we run the mutex watershed algorithm on the full\ngraph to get a globally optimal look up table.", "type": "object", "properties": { "roi": { "maxItems": 2, "minItems": 2, "prefixItems": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "items": { "type": "integer" }, "minItems": 1, "type": "array" } ], "title": "Roi", "type": "array" }, "num_workers": { "default": 1, "title": "Num Workers", "type": "integer" }, "num_cache_workers": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Num Cache Workers" }, "worker_config": { "anyOf": [ { "$ref": "#/$defs/Worker" }, { "type": "null" } ], "default": null }, "fit": { "const": "shrink", "default": "shrink", "title": "Fit", "type": "string" }, "read_write_conflict": { "const": false, "default": false, "title": "Read Write Conflict", "type": "boolean" }, "task_type": { "const": "graph-mws", "default": "graph-mws", "title": "Task Type", "type": "string" }, "db": { "discriminator": { "mapping": { "postgresql": "#/$defs/PostgreSQL", "sqlite": "#/$defs/SQLite" }, "propertyName": "db_type" }, "oneOf": [ { "$ref": "#/$defs/PostgreSQL" }, { "$ref": "#/$defs/SQLite" } ], "title": "Db" }, "lut": { "$ref": "#/$defs/LUT" }, "starting_lut": { "anyOf": [ { "$ref": "#/$defs/LUT" }, { "type": "null" } ], "default": null }, "weights": { "additionalProperties": { "maxItems": 2, "minItems": 2, "prefixItems": [ { "type": "number" }, { "type": "number" } ], "type": "array" }, "title": "Weights", "type": "object" }, "edge_per_attr": { "default": true, "title": "Edge Per Attr", "type": "boolean" }, "mean_attrs": { "anyOf": [ { "additionalProperties": { "type": "string" }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Mean Attrs" }, "out_db": { "anyOf": [ { "discriminator": { "mapping": { "postgresql": "#/$defs/PostgreSQL", "sqlite": "#/$defs/SQLite" }, "propertyName": "db_type" }, "oneOf": [ { "$ref": "#/$defs/PostgreSQL" }, { "$ref": "#/$defs/SQLite" } ] }, { "type": "null" } ], "default": null, "title": "Out Db" }, "bounded_read": { "default": true, "title": "Bounded Read", "type": "boolean" } }, "$defs": { "LUT": { "additionalProperties": false, "description": "A class for defining look up tables", "properties": { "path": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "string" } ], "title": "Path" } }, "required": [ "path" ], "title": "LUT", "type": "object" }, "PostgreSQL": { "additionalProperties": false, "description": "A PostgreSQL database for storing and retrieving graph data.", "properties": { "node_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Node Attrs" }, "edge_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Edge Attrs" }, "ndim": { "default": 3, "title": "Ndim", "type": "integer" }, "db_type": { "const": "postgresql", "default": "postgresql", "title": "Db Type", "type": "string" }, "host": { "default": "localhost", "title": "Host", "type": "string" }, "name": { "default": "volara", "title": "Name", "type": "string" }, "user": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "User" }, "password": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Password" } }, "title": "PostgreSQL", "type": "object" }, "SQLite": { "additionalProperties": false, "description": "An SQLite database for storing and retrieving graph data.", "properties": { "node_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Node Attrs" }, "edge_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Edge Attrs" }, "ndim": { "default": 3, "title": "Ndim", "type": "integer" }, "db_type": { "const": "sqlite", "default": "sqlite", "title": "Db Type", "type": "string" }, "path": { "format": "path", "title": "Path", "type": "string" } }, "required": [ "path" ], "title": "SQLite", "type": "object" }, "Worker": { "additionalProperties": false, "properties": { "queue": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Queue" }, "num_gpus": { "default": 0, "title": "Num Gpus", "type": "integer" }, "num_cpus": { "default": 1, "title": "Num Cpus", "type": "integer" } }, "title": "Worker", "type": "object" } }, "additionalProperties": false, "required": [ "roi", "db", "lut", "weights" ] }
- Config:
extra: str = forbid
- Fields:
bounded_read (bool)db (Annotated[volara.dbs.PostgreSQL | volara.dbs.SQLite, FieldInfo(annotation=NoneType, required=True, discriminator='db_type')])edge_per_attr (bool)fit (Literal['shrink'])lut (volara.lut.LUT)mean_attrs (dict[str, str] | None)out_db (Annotated[volara.dbs.PostgreSQL | volara.dbs.SQLite, FieldInfo(annotation=NoneType, required=True, discriminator='db_type')] | None)read_write_conflict (Literal[False])roi (tuple[volara.utils.PydanticCoordinate, volara.utils.PydanticCoordinate])starting_lut (volara.lut.LUT | None)task_type (Literal['graph-mws'])weights (dict[str, tuple[float, float]])
-
field bounded_read:
bool= True# Reading from the db can be made more efficient by not doing a spatial query and assuming we want all nodes and edges. If you don’t want to process a sub volume of the graph setting this to false will speed up the read.
-
field db:
Annotated[PostgreSQL|SQLite] [Required]#
-
field edge_per_attr:
bool= True# Whether or not to create a separate edge for each attribute in the weights. If False, the sum of all the weighted attributes will be used as the only edge weight.
-
field fit:
Literal['shrink'] = 'shrink'# The strategy to use for blocks that overhang your total write roi.
-
field lut:
LUT[Required]# The Look Up Table that will be saved on completion of this task.
-
field mean_attrs:
dict[str,str] |None= None# A dictionary of attributes to compute the mean of for each segment. Given mean_attrs = {“attr1”: “out_attr1”} and nodes n_i in a segment s we will set s.out_attr1 = sum(n_i.attr1 * n_i.size) / sum(n_i.size).
-
field out_db:
Optional[Annotated[PostgreSQL|SQLite]] = None# The db in which to store segment nodes and their attributes. Must not be None if mean_attrs is not None.
-
field read_write_conflict:
Literal[False] = False# Whether blocks have read/write dependencies on neighborhing blocks requiring a specific ordering to the block processing to compute a seamless result.
-
field roi:
tuple[PydanticCoordinate,PydanticCoordinate] [Required]# The roi to process. This is the roi of the full graph to process.
-
field starting_lut:
LUT|None= None# An optional Look Up Table that provides a set of merged fragments that must be preserved in the final Look Up Table.
-
field task_type:
Literal['graph-mws'] = 'graph-mws'#
-
field weights:
dict[str,tuple[float,float]] [Required]# A dictionary of edge attributes and their weight and bias. These will be used to compute the edge weights for the mutex watershed algorithm. Positive edges will result in fragments merging, negative edges will result in splitting and edges will be processed in order of high to low magnitude. Each attribute will have a final score of w * edge_data[attr] + b for every attr, (w, b) in weights.items() If an attribute is not present in the edge data it will be skipped.
- drop_artifacts()#
A helper function to reset anything produced by a task to a clean state equivalent to not having run the task at all
- model_post_init(context, /)#
This function is meant to behave like a BaseModel method to initialise private attributes.
It takes context as an argument since that’s what pydantic-core passes when calling it.
- Parameters:
self (
BaseModel) – The BaseModel instance.context (
Any) – The context.
- Return type:
None
- process_block_func()#
A constructor for a function that will take a single block as input and process it.
- property context_size: Coordinate#
The amount of context needed to process each block for a task.
- property num_voxels_in_block: int#
- property task_name: str#
A unique identifier for a task. This allows us to store log files in an unambiguous location as well as storing a block_done dataset that allows us to cache completed blocks and resume processing at a later time.
- property write_roi: Roi#
The total roi of any data output by a task.
- property write_size: Coordinate#
The write size of each block processed as part of a task.
- pydantic model volara.blockwise.Relabel#
A task for blockwise relabelling of arrays using a lookup table from fragment to segment IDs.
Show JSON schema
{ "title": "Relabel", "description": "A task for blockwise relabelling of arrays using a lookup table from fragment\nto segment IDs.", "type": "object", "properties": { "roi": { "anyOf": [ { "maxItems": 2, "minItems": 2, "prefixItems": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "items": { "type": "integer" }, "minItems": 1, "type": "array" } ], "type": "array" }, { "type": "null" } ], "default": null, "title": "Roi" }, "num_workers": { "default": 1, "title": "Num Workers", "type": "integer" }, "num_cache_workers": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Num Cache Workers" }, "worker_config": { "anyOf": [ { "$ref": "#/$defs/Worker" }, { "type": "null" } ], "default": null }, "fit": { "const": "shrink", "default": "shrink", "title": "Fit", "type": "string" }, "read_write_conflict": { "const": false, "default": false, "title": "Read Write Conflict", "type": "boolean" }, "task_type": { "const": "relabel", "default": "relabel", "title": "Task Type", "type": "string" }, "frags_data": { "$ref": "#/$defs/Labels" }, "seg_data": { "$ref": "#/$defs/Labels" }, "lut": { "$ref": "#/$defs/LUT" }, "block_size": { "items": { "type": "integer" }, "minItems": 1, "title": "Block Size", "type": "array" } }, "$defs": { "LUT": { "additionalProperties": false, "description": "A class for defining look up tables", "properties": { "path": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "string" } ], "title": "Path" } }, "required": [ "path" ], "title": "LUT", "type": "object" }, "Labels": { "additionalProperties": false, "description": "Represents an integer label dataset.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "labels", "default": "labels", "title": "Dataset Type", "type": "string" } }, "required": [ "store" ], "title": "Labels", "type": "object" }, "Worker": { "additionalProperties": false, "properties": { "queue": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Queue" }, "num_gpus": { "default": 0, "title": "Num Gpus", "type": "integer" }, "num_cpus": { "default": 1, "title": "Num Cpus", "type": "integer" } }, "title": "Worker", "type": "object" } }, "additionalProperties": false, "required": [ "frags_data", "seg_data", "lut", "block_size" ] }
- Config:
extra: str = forbid
- Fields:
block_size (volara.utils.PydanticCoordinate)fit (Literal['shrink'])frags_data (volara.datasets.Labels)lut (volara.lut.LUT)read_write_conflict (Literal[False])seg_data (volara.datasets.Labels)task_type (Literal['relabel'])
-
field block_size:
PydanticCoordinate[Required]#
-
field fit:
Literal['shrink'] = 'shrink'# The strategy to use for blocks that overhang your total write roi.
-
field lut:
LUT[Required]# The path to the lookup table (LUT) that maps fragment IDs to segment IDs.
-
field read_write_conflict:
Literal[False] = False# Whether blocks have read/write dependencies on neighborhing blocks requiring a specific ordering to the block processing to compute a seamless result.
-
field seg_data:
Labels[Required]# The segments dataset to which we write the relabeled segment IDs.
-
field task_type:
Literal['relabel'] = 'relabel'#
- drop_artifacts()#
A helper function to reset anything produced by a task to a clean state equivalent to not having run the task at all
- init()#
Any one time initializations that need to be made before starting a task such as creating dbs and zarrs.
- init_out_array()#
- map_block(block, frags, segs, mapping)#
- model_post_init(context, /)#
This function is meant to behave like a BaseModel method to initialise private attributes.
It takes context as an argument since that’s what pydantic-core passes when calling it.
- Parameters:
self (
BaseModel) – The BaseModel instance.context (
Any) – The context.
- Return type:
None
- process_block_func()#
A constructor for a function that will take a single block as input and process it.
- property context_size: Coordinate#
The amount of context needed to process each block for a task.
- property task_name: str#
A unique identifier for a task. This allows us to store log files in an unambiguous location as well as storing a block_done dataset that allows us to cache completed blocks and resume processing at a later time.
- property voxel_size: Coordinate#
- property write_roi: Roi#
The total roi of any data output by a task.
- property write_size: Coordinate#
The write size of each block processed as part of a task.
- pydantic model volara.blockwise.SeededExtractFrags#
Extract fragments from affinities using a set of skeletons as a supervising signal. Any voxel that intersects with a node placed on a skeleton is guaranteed to be assigned the label of the skeleton it intersects with. The affinities are used to fill out the rest of the segment to get a full volume representation of your skeletons.
Any fragment that does not intersect with a skeleton is discarded.
Show JSON schema
{ "title": "SeededExtractFrags", "description": "Extract fragments from affinities using a set of skeletons as a supervising signal.\nAny voxel that intersects with a node placed on a skeleton is guaranteed to be assigned\nthe label of the skeleton it intersects with. The affinities are used to fill out\nthe rest of the segment to get a full volume representation of your skeletons.\n\nAny fragment that does not intersect with a skeleton is discarded.", "type": "object", "properties": { "roi": { "anyOf": [ { "maxItems": 2, "minItems": 2, "prefixItems": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "items": { "type": "integer" }, "minItems": 1, "type": "array" } ], "type": "array" }, { "type": "null" } ], "default": null, "title": "Roi" }, "num_workers": { "default": 1, "title": "Num Workers", "type": "integer" }, "num_cache_workers": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Num Cache Workers" }, "worker_config": { "anyOf": [ { "$ref": "#/$defs/Worker" }, { "type": "null" } ], "default": null }, "fit": { "const": "shrink", "default": "shrink", "title": "Fit", "type": "string" }, "read_write_conflict": { "const": false, "default": false, "title": "Read Write Conflict", "type": "boolean" }, "task_type": { "const": "seeded-extract-frags", "default": "seeded-extract-frags", "title": "Task Type", "type": "string" }, "affs_data": { "$ref": "#/$defs/Affs" }, "segs_data": { "$ref": "#/$defs/Labels" }, "mask_data": { "anyOf": [ { "$ref": "#/$defs/Labels" }, { "type": "null" } ], "default": null }, "block_size": { "items": { "type": "integer" }, "minItems": 1, "title": "Block Size", "type": "array" }, "context": { "items": { "type": "integer" }, "minItems": 1, "title": "Context", "type": "array" }, "bias": { "items": { "type": "number" }, "title": "Bias", "type": "array" }, "strides": { "anyOf": [ { "items": { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Strides" }, "graph_db": { "discriminator": { "mapping": { "postgresql": "#/$defs/PostgreSQL", "sqlite": "#/$defs/SQLite" }, "propertyName": "db_type" }, "oneOf": [ { "$ref": "#/$defs/PostgreSQL" }, { "$ref": "#/$defs/SQLite" } ], "title": "Graph Db" }, "randomized_strides": { "default": false, "title": "Randomized Strides", "type": "boolean" } }, "$defs": { "Affs": { "additionalProperties": false, "description": "Represents a dataset containing affinities.\nRequires the inclusion of the neighborhood for these\naffinities.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "affs", "default": "affs", "title": "Dataset Type", "type": "string" }, "neighborhood": { "anyOf": [ { "items": { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Neighborhood" } }, "required": [ "store" ], "title": "Affs", "type": "object" }, "Labels": { "additionalProperties": false, "description": "Represents an integer label dataset.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "labels", "default": "labels", "title": "Dataset Type", "type": "string" } }, "required": [ "store" ], "title": "Labels", "type": "object" }, "PostgreSQL": { "additionalProperties": false, "description": "A PostgreSQL database for storing and retrieving graph data.", "properties": { "node_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Node Attrs" }, "edge_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Edge Attrs" }, "ndim": { "default": 3, "title": "Ndim", "type": "integer" }, "db_type": { "const": "postgresql", "default": "postgresql", "title": "Db Type", "type": "string" }, "host": { "default": "localhost", "title": "Host", "type": "string" }, "name": { "default": "volara", "title": "Name", "type": "string" }, "user": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "User" }, "password": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Password" } }, "title": "PostgreSQL", "type": "object" }, "SQLite": { "additionalProperties": false, "description": "An SQLite database for storing and retrieving graph data.", "properties": { "node_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Node Attrs" }, "edge_attrs": { "anyOf": [ { "additionalProperties": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Edge Attrs" }, "ndim": { "default": 3, "title": "Ndim", "type": "integer" }, "db_type": { "const": "sqlite", "default": "sqlite", "title": "Db Type", "type": "string" }, "path": { "format": "path", "title": "Path", "type": "string" } }, "required": [ "path" ], "title": "SQLite", "type": "object" }, "Worker": { "additionalProperties": false, "properties": { "queue": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Queue" }, "num_gpus": { "default": 0, "title": "Num Gpus", "type": "integer" }, "num_cpus": { "default": 1, "title": "Num Cpus", "type": "integer" } }, "title": "Worker", "type": "object" } }, "additionalProperties": false, "required": [ "affs_data", "segs_data", "block_size", "context", "bias", "graph_db" ] }
- Config:
extra: str = forbid
- Fields:
affs_data (volara.datasets.Affs)bias (list[float])block_size (volara.utils.PydanticCoordinate)context (volara.utils.PydanticCoordinate)fit (Literal['shrink'])graph_db (Annotated[volara.dbs.PostgreSQL | volara.dbs.SQLite, FieldInfo(annotation=NoneType, required=True, discriminator='db_type')])mask_data (volara.datasets.Labels | None)randomized_strides (bool)read_write_conflict (Literal[False])segs_data (volara.datasets.Labels)strides (list[volara.utils.PydanticCoordinate] | None)task_type (Literal['seeded-extract-frags'])
-
field affs_data:
Affs[Required]# The affinities dataset that will be used to expand the skeletons to full segments.
-
field bias:
list[float] [Required]# The bias terms to be used for each offset in the affinities neighborhoood.
-
field block_size:
PydanticCoordinate[Required]#
-
field context:
PydanticCoordinate[Required]#
-
field fit:
Literal['shrink'] = 'shrink'# The strategy to use for blocks that overhang your total write roi.
-
field graph_db:
Annotated[PostgreSQL|SQLite] [Required]# The graph database containing the skeletons to use as a supervising signale.
-
field randomized_strides:
bool= False# Whether or not to convert the strides from a grid like filter to a random probability of filtering out each affinity edge.
-
field read_write_conflict:
Literal[False] = False# Whether blocks have read/write dependencies on neighborhing blocks requiring a specific ordering to the block processing to compute a seamless result.
-
field segs_data:
Labels[Required]# The segmentations dataset that will contain the final segmentations.
-
field strides:
list[PydanticCoordinate] |None= None# The strides with which to filter each offset in the affinities neighborhood.
-
field task_type:
Literal['seeded-extract-frags'] = 'seeded-extract-frags'#
- drop_artifacts()#
A helper function to reset anything produced by a task to a clean state equivalent to not having run the task at all
- init()#
Any one time initializations that need to be made before starting a task such as creating dbs and zarrs.
- init_out_array()#
- model_post_init(context, /)#
This function is meant to behave like a BaseModel method to initialise private attributes.
It takes context as an argument since that’s what pydantic-core passes when calling it.
- Parameters:
self (
BaseModel) – The BaseModel instance.context (
Any) – The context.
- Return type:
None
- process_block_func()#
A constructor for a function that will take a single block as input and process it.
- property context_size: Coordinate#
The amount of context needed to process each block for a task.
- property task_name: str#
A unique identifier for a task. This allows us to store log files in an unambiguous location as well as storing a block_done dataset that allows us to cache completed blocks and resume processing at a later time.
- property voxel_size: Coordinate#
- property write_roi: Roi#
The total roi of any data output by a task.
- property write_size: Coordinate#
The write size of each block processed as part of a task.
- pydantic model volara.blockwise.Threshold#
Blockwise threshold an array
Show JSON schema
{ "title": "Threshold", "description": "Blockwise threshold an array", "type": "object", "properties": { "roi": { "anyOf": [ { "maxItems": 2, "minItems": 2, "prefixItems": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "items": { "type": "integer" }, "minItems": 1, "type": "array" } ], "type": "array" }, { "type": "null" } ], "default": null, "title": "Roi" }, "num_workers": { "default": 1, "title": "Num Workers", "type": "integer" }, "num_cache_workers": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Num Cache Workers" }, "worker_config": { "anyOf": [ { "$ref": "#/$defs/Worker" }, { "type": "null" } ], "default": null }, "fit": { "const": "shrink", "default": "shrink", "title": "Fit", "type": "string" }, "read_write_conflict": { "const": false, "default": false, "title": "Read Write Conflict", "type": "boolean" }, "task_type": { "const": "threshold", "default": "threshold", "title": "Task Type", "type": "string" }, "in_data": { "$ref": "#/$defs/Raw" }, "mask": { "$ref": "#/$defs/Labels" }, "threshold": { "title": "Threshold", "type": "number" }, "block_size": { "items": { "type": "integer" }, "minItems": 1, "title": "Block Size", "type": "array" } }, "$defs": { "Dataset": { "additionalProperties": false, "description": "A Dataset base class that defines the common attributes and methods\nfor all dataset types.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" } }, "required": [ "store" ], "title": "Dataset", "type": "object" }, "Labels": { "additionalProperties": false, "description": "Represents an integer label dataset.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "labels", "default": "labels", "title": "Dataset Type", "type": "string" } }, "required": [ "store" ], "title": "Labels", "type": "object" }, "Raw": { "additionalProperties": false, "description": "Represents a dataset containing raw intensities.\nHas support for sampling specific channels, normalizing\nwith provided scale and shifting, or reading in normalization\nbounds from OMERO metadata.", "properties": { "store": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Store" }, "voxel_size": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Voxel Size" }, "offset": { "anyOf": [ { "items": { "type": "integer" }, "minItems": 1, "type": "array" }, { "type": "null" } ], "default": null, "title": "Offset" }, "axis_names": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Axis Names" }, "units": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Units" }, "dataset_type": { "const": "raw", "default": "raw", "title": "Dataset Type", "type": "string" }, "channels": { "anyOf": [ { "items": { "type": "integer" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Channels" }, "ome_norm": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "string" }, { "type": "null" } ], "default": null, "title": "Ome Norm" }, "scale_shift": { "anyOf": [ { "maxItems": 2, "minItems": 2, "prefixItems": [ { "type": "number" }, { "type": "number" } ], "type": "array" }, { "type": "null" } ], "default": null, "title": "Scale Shift" }, "stack": { "anyOf": [ { "$ref": "#/$defs/Dataset" }, { "type": "null" } ], "default": null } }, "required": [ "store" ], "title": "Raw", "type": "object" }, "Worker": { "additionalProperties": false, "properties": { "queue": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Queue" }, "num_gpus": { "default": 0, "title": "Num Gpus", "type": "integer" }, "num_cpus": { "default": 1, "title": "Num Cpus", "type": "integer" } }, "title": "Worker", "type": "object" } }, "additionalProperties": false, "required": [ "in_data", "mask", "threshold", "block_size" ] }
- Config:
extra: str = forbid
- Fields:
block_size (volara.utils.PydanticCoordinate)fit (Literal['shrink'])in_data (volara.datasets.Raw)mask (volara.datasets.Labels)read_write_conflict (Literal[False])task_type (Literal['threshold'])threshold (float)
-
field block_size:
PydanticCoordinate[Required]#
-
field fit:
Literal['shrink'] = 'shrink'# The strategy to use for blocks that overhang your total write roi.
-
field read_write_conflict:
Literal[False] = False# Whether blocks have read/write dependencies on neighborhing blocks requiring a specific ordering to the block processing to compute a seamless result.
-
field task_type:
Literal['threshold'] = 'threshold'#
-
field threshold:
float[Required]# The threshold value to apply to your dataset.
- drop_artifacts()#
A helper function to reset anything produced by a task to a clean state equivalent to not having run the task at all
- init()#
Any one time initializations that need to be made before starting a task such as creating dbs and zarrs.
- init_out_array()#
- model_post_init(context, /)#
This function is meant to behave like a BaseModel method to initialise private attributes.
It takes context as an argument since that’s what pydantic-core passes when calling it.
- Parameters:
self (
BaseModel) – The BaseModel instance.context (
Any) – The context.
- Return type:
None
- process_block_func()#
A constructor for a function that will take a single block as input and process it.
- property context_size: Coordinate#
The amount of context needed to process each block for a task.
- property task_name: str#
A unique identifier for a task. This allows us to store log files in an unambiguous location as well as storing a block_done dataset that allows us to cache completed blocks and resume processing at a later time.
- property voxel_size: Coordinate#
- property write_roi: Roi#
The total roi of any data output by a task.
- property write_size: Coordinate#
The write size of each block processed as part of a task.