TrackerMinerFS

TrackerMinerFS — Abstract base class for filesystem miners

Properties

TrackerDataProvider * data-provider Read / Write / Construct Only
guint processing-pool-ready-limit Read / Write / Construct
guint processing-pool-wait-limit Read / Write / Construct
GFile * root Read / Write / Construct Only
gdouble throttle Read / Write

Signals

void finished Run Last
void finished-root Run Last
gchar* move-file Run Last
gboolean process-file Run Last
gboolean process-file-attributes Run Last
gchar* remove-children Run Last
gchar* remove-file Run Last

Object Hierarchy

    GObject
    ╰── TrackerMiner
        ╰── TrackerMinerFS

Implemented Interfaces

TrackerMinerFS implements GInitable.

Includes

#include <libtracker-miner/tracker-miner.h>

Description

TrackerMinerFS is an abstract base class for miners that collect data from a filesystem where parent/child relationships need to be inserted into the database correctly with queue management.

All the filesystem crawling and monitoring is abstracted away, leaving to implementations the decisions of what directories/files should it process, and the actual data extraction.

Example creating a TrackerMinerFS with our own file system root and data provider.

First create our class and base it on TrackerMinerFS:

1
2
3
G_DEFINE_TYPE_WITH_CODE (MyMinerFiles, my_miner_files, TRACKER_TYPE_MINER_FS,
                         G_IMPLEMENT_INTERFACE (G_TYPE_INITABLE,
                                                my_miner_files_initable_iface_init))

Later in our class creation function, we are supplying the arguments we want. In this case, the 'root' is a GFile pointing to a root URI location (for example 'file:///') and 'data_provider' is a TrackerDataProvider used to enumerate 'root' and return children it finds. If 'data_provider' is NULL (the default), then a TrackerFileDataProvider is created automatically.

1
2
3
4
5
6
7
8
9
10
// Note that only 'name' is mandatory
miner = g_initable_new (MY_TYPE_MINER_FILES,
                        NULL,
                        error,
                        "name", "MyMinerFiles",
                        "root", root,
                        "data-provider", data_provider,
                        "processing-pool-wait-limit", 10,
                        "processing-pool-ready-limit", 100,
                        NULL);

Functions

tracker_miner_fs_error_quark ()

GQuark
tracker_miner_fs_error_quark (void);

Gives the caller the GQuark used to identify TrackerMinerFS errors in GError structures. The GQuark is used as the domain for the error.

Returns

the GQuark used for the domain of a GError.

Since: 1.2.


tracker_miner_fs_get_indexing_tree ()

TrackerIndexingTree *
tracker_miner_fs_get_indexing_tree (TrackerMinerFS *fs);

Returns the TrackerIndexingTree which determines what files/directories are indexed by fs

Parameters

fs

a TrackerMinerFS

 

Returns

The TrackerIndexingTree holding the indexing configuration.

[transfer none]


tracker_miner_fs_get_data_provider ()

TrackerDataProvider *
tracker_miner_fs_get_data_provider (TrackerMinerFS *fs);

Returns the TrackerDataProvider implementation, which is being used to supply GFile and GFileInfo content to Tracker.

Parameters

fs

a TrackerMinerFS

 

Returns

The TrackerDataProvider supplying content.

[transfer none]

Since: 1.2


tracker_miner_fs_get_throttle ()

gdouble
tracker_miner_fs_get_throttle (TrackerMinerFS *fs);

Gets the current throttle value, see tracker_miner_fs_set_throttle() for more details.

Parameters

fs

a TrackerMinerFS

 

Returns

a double representing a value between 0.0 and 1.0.

Since: 0.8


tracker_miner_fs_set_throttle ()

void
tracker_miner_fs_set_throttle (TrackerMinerFS *fs,
                               gdouble throttle);

Tells the filesystem miner to throttle its operations. A value of 0.0 means no throttling at all, so the miner will perform operations at full speed, 1.0 is the slowest value. With a value of 1.0, the fs is typically waiting one full second before handling the next batch of queued items to be processed.

Parameters

fs

a TrackerMinerFS

 

throttle

a double between 0.0 and 1.0

 

Since: 0.8


tracker_miner_fs_check_file ()

void
tracker_miner_fs_check_file (TrackerMinerFS *fs,
                             GFile *file,
                             gint priority,
                             gboolean check_parents);

Tells the filesystem miner to check and index a file at a given priority, this file must be part of the usual crawling directories of TrackerMinerFS. See tracker_miner_fs_directory_add().

Parameters

fs

a TrackerMinerFS

 

file

GFile for the file to check

 

priority

the priority of the check task

 

check_parents

whether to check parents and eligibility or not

 

Since: 0.10


tracker_miner_fs_notify_finish ()

void
tracker_miner_fs_notify_finish (TrackerMinerFS *fs,
                                GTask *task,
                                const gchar *sparql,
                                GError *error);

Notifies fs that all processing on file has been finished, if any error happened during file data processing, it should be passed in error , else sparql should contain correct SPARQL representing the operation in particular.

This function is expected to be called in reaction to all TrackerMinerFS signals

Parameters

fs

a TrackerMinerFS

 

task

a GTask obtained in a TrackerMinerFS signal/vmethod

 

sparql

Resulting sparql for the given operation, or NULL if there is an error.

[nullable]

error

a GError with the error that happened during processing, or NULL.

 

tracker_miner_fs_get_urn ()

const gchar *
tracker_miner_fs_get_urn (TrackerMinerFS *fs,
                          GFile *file);

If the item exists in the store, this function retrieves the URN for a GFile being currently processed.

If file is not being currently processed by fs , or doesn't exist in the store yet, NULL will be returned.

Parameters

fs

a TrackerMinerFS

 

file

a GFile obtained in “process-file”

 

Returns

The URN containing the data associated to file , or NULL.

[transfer none][nullable]

Since: 0.8


tracker_miner_fs_query_urn ()

gchar *
tracker_miner_fs_query_urn (TrackerMinerFS *fs,
                            GFile *file);

If the item exists in the store, this function retrieves the URN of the given GFile

If file doesn't exist in the store yet, NULL will be returned.

Parameters

fs

a TrackerMinerFS

 

file

a GFile

 

Returns

A newly allocated string with the URN containing the data associated to file , or NULL.

[transfer full]

Since: 0.10


tracker_miner_fs_has_items_to_process ()

gboolean
tracker_miner_fs_has_items_to_process (TrackerMinerFS *fs);

The fs keeps many priority queus for content it is processing. This function returns TRUE if the sum of all (or any) priority queues is more than 0. This includes items deleted, created, updated, moved or being written back.

Parameters

fs

a TrackerMinerFS

 

Returns

TRUE if there are items to process in the internal queues, otherwise FALSE.

Since: 0.10

Types and Values

struct TrackerMinerFS

struct TrackerMinerFS;

Abstract miner implementation to get data from the filesystem.


TrackerMinerFSClass

typedef struct {
	TrackerMinerClass parent;

	gboolean (* process_file)             (TrackerMinerFS       *fs,
	                                       GFile                *file,
					       GTask                *task);
	void     (* finished)                 (TrackerMinerFS       *fs,
	                                       gdouble               elapsed,
	                                       gint                  directories_found,
	                                       gint                  directories_ignored,
	                                       gint                  files_found,
	                                       gint                  files_ignored);
	gboolean (* process_file_attributes)  (TrackerMinerFS       *fs,
	                                       GFile                *file,
					       GTask                *task);
	void     (* finished_root)            (TrackerMinerFS       *fs,
	                                       GFile                *root,
	                                       gint                  directories_found,
	                                       gint                  directories_ignored,
	                                       gint                  files_found,
	                                       gint                  files_ignored);
	gchar *  (* remove_file)              (TrackerMinerFS       *fs,
	                                       GFile                *file);
	gchar *  (* remove_children)          (TrackerMinerFS       *fs,
	                                       GFile                *file);
	gchar *  (* move_file)                (TrackerMinerFS       *fs,
					       GFile                *source,
	                                       GFile                *dest,
	                                       gboolean              recursive);

	gboolean (* filter_event)             (TrackerMinerFS          *fs,
	                                       TrackerMinerFSEventType  type,
	                                       GFile                   *file,
	                                       GFile                   *source_file);

	/* <Private> */
	gpointer padding[20];
} TrackerMinerFSClass;

Prototype for the abstract class, process_file must be implemented in the deriving class in order to actually extract data.

Members

TrackerMinerClass parent;

parent object class

 

process_file ()

Called when the metadata associated to a file is requested.

 

finished ()

Called when all processing has been performed.

 

process_file_attributes ()

Called when the metadata associated with a file's attributes changes, for example, the mtime.

 

finished_root ()

Called when all resources on a particular root URI have been processed.

 

remove_file ()

   

remove_children ()

   

move_file ()

   

filter_event ()

   

gpointer padding[20];

Reserved for future API improvements.

 

enum TrackerMinerFSError

Possible errors returned when calling creating new objects based on the TrackerMinerFS type and other APIs available with this class.

Members

TRACKER_MINER_FS_ERROR_INIT

There was an error during initialization of the object. The specific details are in the message.

 

Since: 1.2.


enum TrackerMinerFSEventType

Members

TRACKER_MINER_FS_EVENT_CREATED

   

TRACKER_MINER_FS_EVENT_UPDATED

   

TRACKER_MINER_FS_EVENT_DELETED

   

TRACKER_MINER_FS_EVENT_MOVED

   

Property Details

The “data-provider” property

  “data-provider”            TrackerDataProvider *

Data provider populating data, e.g. like GFileEnumerator.

Flags: Read / Write / Construct Only


The “processing-pool-ready-limit” property

  “processing-pool-ready-limit” guint

Maximum number of SPARQL updates that can be merged in a single connection to the store.

Flags: Read / Write / Construct

Allowed values: >= 1

Default value: 1


The “processing-pool-wait-limit” property

  “processing-pool-wait-limit” guint

Maximum number of files that can be concurrently processed by the upper layer.

Flags: Read / Write / Construct

Allowed values: >= 1

Default value: 1


The “root” property

  “root”                     GFile *

Top level URI for our indexing tree and file notify clases.

Flags: Read / Write / Construct Only


The “throttle” property

  “throttle”                 gdouble

Modifier for the indexing speed, 0 is max speed.

Flags: Read / Write

Allowed values: [0,1]

Default value: 0

Signal Details

The “finished” signal

void
user_function (TrackerMinerFS *miner_fs,
               gdouble         elapsed,
               guint           directories_found,
               guint           directories_ignored,
               guint           files_found,
               guint           files_ignored,
               gpointer        user_data)

The ::finished signal is emitted when miner_fs has finished all pending processing.

Parameters

miner_fs

the TrackerMinerFS

 

elapsed

elapsed time since mining was started

 

directories_found

number of directories found

 

directories_ignored

number of ignored directories

 

files_found

number of files found

 

files_ignored

number of ignored files

 

user_data

user data set when the signal handler was connected.

 

Flags: Run Last

Since: 0.8


The “finished-root” signal

void
user_function (TrackerMinerFS *miner_fs,
               GFile          *file,
               gpointer        user_data)

The ::finished-crawl signal is emitted when miner_fs has finished finding all resources that need to be indexed with the root location of file . At this point, it's likely many are still in the queue to be added to the database, but this gives some indication that a location is processed.

Parameters

miner_fs

the TrackerMinerFS

 

file

a GFile

 

user_data

user data set when the signal handler was connected.

 

Flags: Run Last

Since: 1.2


The “move-file” signal

gchar*
user_function (TrackerMinerFS *trackerminerfs,
               GFile          *arg1,
               GFile          *arg2,
               gboolean        arg3,
               gpointer        user_data)

Flags: Run Last


The “process-file” signal

gboolean
user_function (TrackerMinerFS *miner_fs,
               GFile          *file,
               GTask          *builder,
               gpointer        user_data)

The ::process-file signal is emitted whenever a file should be processed, and it's metadata extracted.

builder is the TrackerSparqlBuilder where all sparql updates to be performed for file will be appended.

This signal allows both synchronous and asynchronous extraction, in the synchronous case cancellable can be safely ignored. In either case, on successful metadata extraction, implementations must call tracker_miner_fs_notify_finish() to indicate that processing has finished on file , so the miner can execute the SPARQL updates and continue processing other files.

Parameters

miner_fs

the TrackerMinerFS

 

file

a GFile

 

builder

a TrackerSparqlBuilder

 

cancellable

a GCancellable

 

user_data

user data set when the signal handler was connected.

 

Returns

TRUE if the file is accepted for processing, FALSE if the file should be ignored.

Flags: Run Last

Since: 0.8


The “process-file-attributes” signal

gboolean
user_function (TrackerMinerFS *miner_fs,
               GFile          *file,
               GTask          *builder,
               gpointer        user_data)

The ::process-file-attributes signal is emitted whenever a file should be processed, but only the attribute-related metadata extracted.

builder is the TrackerSparqlBuilder where all sparql updates to be performed for file will be appended. For the properties being updated, the DELETE statements should be included as well.

This signal allows both synchronous and asynchronous extraction, in the synchronous case cancellable can be safely ignored. In either case, on successful metadata extraction, implementations must call tracker_miner_fs_notify_finish() to indicate that processing has finished on file , so the miner can execute the SPARQL updates and continue processing other files.

Parameters

miner_fs

the TrackerMinerFS

 

file

a GFile

 

builder

a TrackerSparqlBuilder

 

cancellable

a GCancellable

 

user_data

user data set when the signal handler was connected.

 

Returns

TRUE if the file is accepted for processing, FALSE if the file should be ignored.

Flags: Run Last

Since: 0.10


The “remove-children” signal

gchar*
user_function (TrackerMinerFS *trackerminerfs,
               GFile          *arg1,
               gpointer        user_data)

Flags: Run Last


The “remove-file” signal

gchar*
user_function (TrackerMinerFS *miner_fs,
               GFile          *file,
               gpointer        user_data)

The ::remove-file signal will be emitted on files that need removal according to the miner configuration (either the files themselves are deleted, or the directory/contents no longer need inspection according to miner configuration and their location.

This operation is always assumed to be recursive, the children_only argument will be TRUE if for any reason the topmost directory needs to stay (e.g. moved from a recursively indexed directory tree to a non-recursively indexed location).

The builder argument can be used to provide additional SPARQL deletes and updates necessary around the deletion of those items. If the return value of this signal is TRUE, builder is expected to contain all relevant deletes for this operation.

If the return value of this signal is FALSE, the miner will apply its default behavior, which is deleting all triples that correspond to the affected URIs.

Parameters

miner_fs

the TrackerMinerFS

 

file

a GFile

 

children_only

TRUE if only the children of file are to be deleted

 

builder

a TrackerSparqlBuilder

 

user_data

user data set when the signal handler was connected.

 

Returns

TRUE if builder contains all the necessary operations to delete the affected resources, FALSE to let the miner implicitly handle the deletion.

Flags: Run Last

Since: 1.8