SQL Connector

Examples

This example implements the use of SQL Connector. The example above use mysql as a database and to wok with this RDBMS you don’t have to install any other libraries, but if you want to work if SQLite, PostgreSQL you should install the libraries for framework enrich properly without errors. You can see all RDBMS available here: SQLAlchemy.

You have an option to pass the engine already created instead connection_url. See the especifications on the docstrings of SQLConnector class.

Besides that, this example use a table named apps and a column to query named #id, and the result of enrichment is placed on attribute named apps. The dict_keys is util to reach at the data that will be used to make the relationship of your original daa and the data stored in database.

# name of file: enrich-sql.py

URL = "mysql+pymysql://root:root@localhost:3306/database"

from moredata.enricher import EnricherBuilder, Enricher
from moredata.enricher.sql_connector import SqlConnector
from moredata.models.data import Data
from moredata.parser import parse_document
from moredata.utils.util import read_json_from_file, Converter

user = Data(data_file=USER_DATA, parser_func=parse_document, data_type="json")

sql_enricher = Enricher(connector=SqlConnector(connection_url=URL, table_name="apps", column='#id', result_attr="apps", dict_keys=["applications_id_list"]))

user_enriched = \
      EnricherBuilder(user) \
      .with_enrichment(sql_enricher) \
      .get_result()

import moredata.utils.util as util

util.write_json_generator_to_json("../../data/output/sql/test", user_enriched, 100000)

$ python enrich-sql.py

SQL Connector

class moredata.enricher.sql_connector.sql_connector.SqlConnector(table_name, dict_keys, result_attr, column, engine=None, connection_url=None)

SQLconnector implements interface IEnricherConnector, so this is a connector that can be used to enrich data.

Parameters

table_name (str) – name of table that will be used to enrich
dict_keys (List[str]) – dict_keys is the principal argument to the connector. The connector doesn’t know where to get data to make the relationship, so you have to pass the keys or if your data isn’t nested, just one key to the connector reach at the the right attribute.
result_attr (str) – where tha enriched data it supposed to be
column (str) – name of table column to build the relation.
engine (sqlalchemy.engine.Engine) – an engine already created by sqlalchemy
connection_url (str) – the string of connection with host, port, password and database.

table_name

Type: str

dict_keys

Type: List

result_attr

Type: str

column

Type: str

engine

Type: sqlalchemy.engine.Engine, optional

connection_url

Type: str, optional

enrich(data, **kwargs)

Method overrided of interface. This method do enrichment using RDBMS as a enricher. It walk through the keys to reach at the data that will be used to create the relationship. After, if the object is a list it creates an attribute on parent object, if the object is just a dict it creates an attribute inside.

Parameters: data (Data) –