Indexing

sir.indexing._index_entity_process_wrapper(args, live=False)[source]

Calls sir.indexing.index_entity() with args unpacked.

Parameters:

live (bool)

Return type:

None or an Exception

sir.indexing._multiprocessed_import(entity_names, live=False, entities=None)[source]

Does the real work to import all entities with entity_name in multiple processes via the multiprocessing module.

When live is True, it means, we are live indexing documents with ids in the entities dict, otherwise it reindexes the entire table for entities in entity_names.

Parameters:
sir.indexing.index_entity(session, entity_name, bounds, data_queue)[source]

Retrieve rows for a single entity type identified by entity_name, convert them to a dict with sir.indexing.query_result_to_dict() and put the dicts into queue.

Parameters:
sir.indexing.live_index(entities)[source]

Reindex all documents in``entities`` in multiple processes via the multiprocessing module.

Parameters:

entities (dict(set(int)))

sir.indexing.live_index_entity(session, entity_name, ids, data_queue)[source]

Retrieve rows for a single entity type identified by entity_name, convert them to a dict with sir.indexing.query_result_to_dict() and put the dicts into queue.

Parameters:
sir.indexing.queue_to_solr(queue, batch_size, entity_name)[source]

Read dict objects from queue and send them to the Solr server behind solr_connection in batches of batch_size.

Parameters:
sir.indexing.reindex(args)[source]

Reindexes all entity types in args[“entity_type”].

If no types are specified, all known entities will be reindexed.

Parameters:

args (dict) – A dictionary with a key named entities.

sir.indexing.send_data_to_solr(solr_connection, data)[source]

Sends data through solr_connection.

Parameters:
Raises:

solr.SolrException