neo package

Submodules

neo.neo4j_tools module

class vfb_connect.neo.neo4j_tools.Neo4jConnect(endpoint='http://pdb.virtualflybrain.org', usr='neo4j', pwd='vfb')

Bases: object

Thin layer over REST API to hold connection details, handle multi-statement POST queries, return results and report errors.

Parameters:

• endpoint – a neo4j REST endpoint

• usr – username (content ignored if credentials not rqd)

• pwd – password (content ignored if credentials not rqd)

commit_csv(url, statement, chunk_size=1000, sep=',')

commit_list(statements, return_graphs=False)

Commit a list of statements to neo4J DB via REST API. Errors prompt warnings (STDERR), not exceptions, and cause return = FALSE.

Param:

statements: A list of cypher statements.

Param:

return_graphs: Optional. Boolean. Returns graphs under ‘graph’ key if True. Default: False

Return:

List of results or False if any errors are encountered.

commit_list_in_chunks(statements, verbose=False, chunk_length=1000)

Commit multiple (chunked) commit of statements to neo4J DB via REST API. Errors prompt warnings (STDOUT), not exceptions, and cause return = FALSE.

Param:

statements: A list of cypher statements.

Param:

verbose: Boolean. Optionally print periodic reports of progress to STDOUT

Param:

chunk_length. Int. Optional. Set chunk size. Default = 1000.

Return:

List of results or False if any errors are encountered. Chunking is not reflected in results.

construct_where_clause(prefixes, base_clause='')

Construct the WHERE clause for the query based on term type and prefix limitations.

Parameters:

• prefixes – List of id prefixes for limiting lookup.

• base_clause – Additional base condition for the WHERE clause.

Returns:

A string representing the WHERE clause for the query.

execute_and_process_query(term_type, where, property_name, out, verbose)

Execute a Cypher query and process the results.

Parameters:

• term_type – The type of terms to fetch (e.g., ‘Class’, ‘Individual’, ‘ObjectProperty’).

• where – The WHERE clause of the query.

• property_name – The property to fetch (e.g., ‘label’, ‘symbol[0]’).

• out – The list to store fetched terms.

• verbose – If True, provides verbose output.

execute_and_process_query_with_synonyms(term_type, where, out, verbose)

Execute a Cypher query to fetch terms and their synonyms.

Parameters:

• term_type – The type of terms to fetch (e.g., ‘Class’, ‘Individual’).

• where – The WHERE clause of the query.

• out – The list to store fetched terms.

• verbose – If True, provides verbose output.

get_lookup(cache=None, limit_type_by_prefix=('FBbt', 'VFBexp', 'VFBext'), include_individuals=True, limit_properties_by_prefix=('RO', 'BFO', 'VFBext'), curies=False, include_synonyms=True, verbose=False)

Generate a name:ID lookup from a VFB neo4j DB, optionally restricted by a list of prefixes.

Parameters:

• cache – If a valid cache path is provided, uses a cached lookup. Otherwise, generates a new lookup.

• limit_type_by_prefix – Optional list of id prefixes for limiting lookup of classes & individuals.

• include_individuals – If True, individuals are included in the lookup.

• limit_properties_by_prefix – Optional list of id prefixes for limiting lookup of properties.

• curies – If True, returns CURIEs instead of IDs.

• include_synonyms – If True, includes synonyms in the lookup.

• verbose – If True, provides verbose output.

Returns:

A dictionary with names (or synonyms) as keys and their corresponding IDs as values.

list_all_edge_props()

list_all_node_props()

load_classes(out, limit_type_by_prefix, include_synonyms, verbose)

Load Class terms into the output list.

Parameters:

• out – The list to store fetched terms.

• limit_type_by_prefix – Optional list of id prefixes for limiting lookup of classes.

• include_synonyms – If True, includes synonyms in the lookup.

• verbose – If True, provides verbose output.

load_individuals(out, include_synonyms, verbose)

Load Individual terms into the output list.

Parameters:

• out – The list to store fetched terms.

• include_synonyms – If True, includes synonyms in the lookup.

• verbose – If True, provides verbose output.

load_object_properties(out, limit_properties_by_prefix, verbose)

Load ObjectProperties into the output list.

Parameters:

• out – The list to store fetched terms.

• limit_properties_by_prefix – Optional list of id prefixes for limiting lookup of properties.

• verbose – If True, provides verbose output.

process_and_add_results(query_result, out, verbose)

Process the results of a query and add to the output list, checking for duplicates.

Parameters:

• query_result – Result set from the query.

• out – The list to store fetched terms.

• verbose – If True, provides verbose output.

process_results(out, curies)

Remove duplicates and prepare the final lookup dictionary.

Parameters:

• out – List of results to process.

• curies – If True, convert IDs to CURIE format.

Returns:

Final lookup dictionary.

rest_return_check(response)

Checks status response to post. Prints warnings if not OK. If OK, checks for errors in response. Prints any present as warnings. Returns True STATUS OK and no errors, otherwise returns False.

save_to_cache(lookup, cache)

Save the lookup to a cache file if caching is enabled.

Parameters:

• lookup – The lookup dictionary to save.

• cache – The cache file path.

test_connection()

Test neo4j endpoint connection

vfb_connect.neo.neo4j_tools.chunks(l, n)

Yield successive n-sized chunks from l.

Parameters:

l – a list

Param:

n, chunk size

vfb_connect.neo.neo4j_tools.cli_credentials()

Parses command line credentials for Neo4J rest connection; Optionally specify additional args as a list of dicts with args required by argparse.add_argument(). Order in list specified arg order

vfb_connect.neo.neo4j_tools.cli_neofj_connect()

vfb_connect.neo.neo4j_tools.dict_cursor(results)

Takes JSON results from a neo4J query and turns them into a list of dicts.

Parameters:

results – neo4j query results

Returns:

list of dicts

vfb_connect.neo.neo4j_tools.escape_string(strng)

Simple escaping for strings used in neo queries.

neo.query_wrapper module

class vfb_connect.neo.query_wrapper.QueryWrapper(*args, **kwargs)

Bases: Neo4jConnect

get_DataSet_TermInfo(short_forms, summary=True, return_dataframe=True)

Generate JSON reports for datasets from a list of VFB IDs (short_forms) of datasets.

Parameters:

• short_forms – An iterable (e.g. a list) of VFB IDs (short_forms) of datasets

• summary – Optional. Returns summary reports if True. Default True

Return type:

list of VFB_json or summary_report_json

get_TermInfo(short_forms, summary=True, cache=True, return_dataframe=True, limit=None, verbose=False)

Generate a JSON report or summary for terms specified by a list of VFB IDs.

This method retrieves term information for a list of specified VFB IDs (short_forms). It can return either full metadata or a summary of the terms. The results can be returned as a pandas DataFrame if return_dataframe is set to True.

Parameters:

• short_forms (iter) – An iterable (e.g., a list) of VFB IDs (short_forms).

• summary – Optional. If True, returns a summary report instead of full metadata. Default is True.

• cache – Optional. If True, attempts to retrieve cached results before querying. Default is True.

• return_dataframe – Optional. If True, returns the results as a pandas DataFrame. Default is True.

Returns:

A list of term metadata as VFB_json or summary_report_json, or a pandas DataFrame if return_dataframe is True.

Return type:

list of dicts or pandas.DataFrame

get_anatomical_individual_TermInfo(short_forms, summary=True, return_dataframe=True)

Generate JSON reports for anatomical individuals from a list of VFB IDs (short_forms)

Parameters:

• short_forms – An iterable (e.g. a list) of VFB IDs (short_forms) of anatomical individuals

• summary – Optional. Returns summary reports if True. Default True

Return type:

list of VFB_json or summary_report_json

get_anatomy_by_type_TermInfo(short_forms, summary=True, return_dataframe=True)

Generate JSON reports for anatomical individuals by type from a list of VFB IDs (short_forms).

Parameters:

• short_forms – An iterable (e.g. a list) of VFB IDs (short_forms) of anatomical types

• summary – Optional. Returns summary reports if True. Default True

Return type:

list of VFB_json or summary_report_json

get_datasets(summary=True, return_dataframe=True)

Generate a report of all datasets available in the system.

This method retrieves all datasets and returns their information as either full JSON data structures or summaries. The results can be returned as a list of dictionaries or as a pandas DataFrame, depending on the return_dataframe flag.

Parameters:

• summary – Optional. If True, returns summary reports instead of full metadata. Default is True.

• return_dataframe – Optional. If True and summary is also True, returns the results as a pandas DataFrame. Default is True.

Returns:

A list of terms as nested Python data structures (VFB_json or summary_report_json), or a pandas DataFrame if return_dataframe is True and summary is True.

Return type:

list of dicts or pandas.DataFrame

get_dbs(include_symbols=False)

Get a list of available database IDs

Returns:

list of VFB IDs.

get_images(short_forms, template, image_folder, image_type='swc', stomp=False)

Given an iterable of short_forms for instances, find all images of specified image_type registered to template. Save these to image_folder along with a manifest.tsv. Return manifest as pandas DataFrame.

Parameters:

• short_forms (iter) – iterable (e.g. list) of VFB IDs of Individuals with images

• template – template name

• image_folder – folder to save image files & manifest to.

• image_type – image type (file extension)

• stomp – Overwrite image_folder if already exists.

Returns:

Manifest as Pandas DataFrame

get_images_by_filename(filenames, dataset=None, summary=True, return_dataframe=True)

Takes a list of filenames as input and returns a list of image terminfo. Optionally restrict by dataset (improves speed)

get_license_TermInfo(short_forms, summary=True, return_dataframe=True)

Generate JSON reports for licenses from a list of VFB IDs (short_forms) of licenses.

Parameters:

• short_forms – An iterable (e.g. a list) of VFB IDs (short_forms) of licenses

• summary – Optional. Returns summary reports if True. Default True

Return type:

list of VFB_json or summary_report_json

get_neuron_class_TermInfo(short_forms, summary=True, return_dataframe=True)

Generate JSON reports for neuron classes from a list of VFB IDs (short_forms) of neuron classes.

Parameters:

• short_forms – An iterable (e.g. a list) of VFB IDs (short_forms) of neuron classes

• summary – Optional. Returns summary reports if True. Default True

Return type:

list of VFB_json or summary_report_json

get_pub_TermInfo(short_forms, summary=True, return_dataframe=True)

Generate JSON reports for publications from a list of VFB IDs (short_forms) of publications.

Parameters:

• short_forms – An iterable (e.g. a list) of VFB IDs (short_forms) of publications

• summary – Optional. Returns summary reports if True. Default True

Return type:

list of VFB_json or summary_report_json

get_split_class_TermInfo(short_forms, summary=True, return_dataframe=True)

Generate JSON reports for split classes from a list of VFB IDs (short_forms) of split classes.

Parameters:

• short_forms – An iterable (e.g. a list) of VFB IDs (short_forms) of split classes

• summary – Optional. Returns summary reports if True. Default True

Return type:

list of VFB_json or summary_report_json

get_template_TermInfo(short_forms, summary=True, return_dataframe=True)

Generate JSON reports for templates from a list of VFB IDs (short_forms) of templates.

Parameters:

• short_forms – An iterable (e.g. a list) of VFB IDs (short_forms) of templates

• summary – Optional. Returns summary reports if True. Default True

Return type:

list of VFB_json or summary_report_json

get_templates(summary=True, return_dataframe=True)

Generate JSON report of all available templates.

Parameters:

summary – Optional. Returns summary reports if True. Default True

Returns:

Returns a list of terms as nested python data structures following VFB_json or a summary_report_json

Return type:

list of VFB_json or summary_report_json

get_terms_by_xref(acc, db='', id_type='', summary=True, return_dataframe=True)

Generate a JSON report for terms specified by a list of cross-reference IDs (xrefs).

This method maps external database IDs (xrefs) to VFB (Virtual Fly Brain) IDs and then retrieves detailed term information for those VFB IDs. The information can be returned either as a summary or as full term metadata.

Parameters:

• acc – An iterable (e.g., a list) of external cross-reference IDs.

• db – Optional. Specify the VFB ID (short_form) of an external database to map to. (Use get_dbs() to find options).

• id_type – Optional. Specify an external ID type to filter the mapping results.

• summary – Optional. Returns summary reports if True. Default is True.

• return_dataframe – Optional. Includes related datasets in the report if True. Default is True.

Returns:

A list of term metadata as VFB_json or summary_report_json.

Return type:

list of dicts

Raises:

Warning – If no VFB ID is found for a given xref.

get_type_TermInfo(short_forms, summary=True, return_dataframe=True)

Generate JSON reports for types from a list of VFB IDs (short_forms) of classes/types.

Parameters:

• short_forms – An iterable (e.g. a list) of VFB IDs (short_forms) of types

• summary – Optional. Returns summary reports if True. Default True

Return type:

list of VFB_json or summary_report_json

vfb_id_2_neuprint_bodyID(vfb_id, db='')

vfb_id_2_xrefs(vfb_id, db='', id_type='', reverse_return=False)

Map a list of short_form IDs in VFB to external DB IDs

Parameters:

• vfb_id (iter) – An iterable (e.g. a list) of VFB short_form IDs.

• db – optional specify the VFB id (short_form) of an external DB to map to. (use get_dbs to find options)

• id_type – optionally specify an external id_type

• reverse_return – Boolean: Optional (see return)

Returns:

if reverse_return is False:

dict { VFB_id : [{ db: <db> : acc : <acc> }

Return if reverse_return is True:

dict { acc : [{ db: <db> : vfb_id : <VFB_id> }

xref_2_vfb_id(acc=None, db='', id_type='', reverse_return=False, verbose=False)

Map a list external DB IDs to VFB IDs

Parameters:

• acc – An iterable (e.g. a list) of external IDs (e.g. neuprint bodyIDs).

• db – optional specify the VFB id (short_form) of an external DB to map to. (use get_dbs to find options)

• id_type – optionally specify an external id_type

• reverse_return – Boolean: Optional (see return)

Returns:

if reverse_return is False:

dict { acc : [{ db: <db> : vfb_id : <VFB_id> }

Return if reverse_return is True:

dict { VFB_id : [{ db: <db> : acc : <acc> }

vfb_connect.neo.query_wrapper.batch_query(func)

Decorator to batch the first argument of the function (assumed to be an iterable) and apply the function to each batch.

Assumes the first argument is to be batched and that the return value is a list. Only works on class methods.

Parameters:

func – The function to be wrapped and batched.

Returns:

A wrapper function that processes the input in batches.

vfb_connect.neo.query_wrapper.gen_simple_report(terms, dataframe=True)

Module contents