Databases

Another common source of data is a relational database, a structure that provides a mechanism to efficiently store, access, and manipulate your structured data. You fetch from or send a portion of data to tables in the database using a Structured Query Language (SQL) request. For instance, the following request issued to an employees table in the database retrieves the list of only those programmers who work in the IT department, making it unnecessary to fetch the entire table:

SELECT first_name, last_name FROM employees WHERE department= 'IT' and title = 'programmer'

Python has a built-in database engine, SQLite. Alternatively, you can employ any other available database. Before you can access a database, you’ll need to install the database client software in your environment.

In addition to the conventional rigidly structured databases, there’s been an ever-increasing need in recent years for the ability to store heterogeneous and unstructured data in database-like containers. This has led to the rise of so-called NoSQL (non-SQL or not only SQL) databases. NoSQL databases use flexible data models, allowing you to store large volumes of unstructured data using the key-value method, where each piece of data can be accessed using an associated key. Here’s what our earlier sample financial statement might look like if stored in a NoSQL database:

key value

--- -----

...

26  GoodComp shares soared as much as 8.2% on 2021-01-07 after the company announced ...

The entire statement is paired with an identifying key, 26. It might seem odd to store the entire statement in a database. Recall, however, that several possible records can be extracted from a single statement. Storing the whole statement gives us the flexibility to extract different pieces of information at a later time.

In the next post we shall look into another source which is Files