Using MongoDB with python (using PyMongo)
h
TheDrone7 (597)

How to use a MongoDB database with your python repl! (PyMongo)

Why though?

One of the most common uses of repl.it is to create discord bots and the most common way to store information like "user warnings" is to store it in files (JSON or even txt). But due to some reasons, repl.it does not save the changes made to these files programmatically, so one great alternative would be to use an external DB and as it stands, MongoDB is one of the easiest to learn and most efficient database services out there and PyMongo is an amazing library that allows you to store and retrieve and perform other functions with your MongoDB database in a really friendly way. So, that's why.

Installation

Although, if you were doing it locally, you'd need to open up a command line utility and use the following command: -

$ python -m pip install pymongo

But repl.it makes it even easier allowing you to simply add packages using the package manager or by creating a requirements.txt file! We'll be using requirements.txt file for this tutorial. Feel free to check out the package manager as well.

If you don't already know about it, all you need to know about the requirements.txt file is that it tells the interpreter which packages to install before running the program. And all the packages are named on different lines. For example, if we needed to use discord.py and pymongo and flask in a project, the requirements.txt would look something like this: -

discord.py
pymongo
flask

For now, we only need pymongo (in this tutorial) so get rid of the other two unless you need them ( for your own app ).
And we're done installing!


Setting up the Database

Goto the MongoDB website and sign-in (or register).
Once signed-in, you should see a page as shown below: -

Click on the Build a Cluster button which will eventually show you a page as shown below: -

Choose your provider, region, plan and other stuff you might wanna have. If you don't know what something means, search on Google (or DDG) or leave it be.

Be sure to give it a name of your choice, nobody wants their DB to be named Cluster0.

Finally, click the Create Cluster button.
And now you'll need to wait for around 5 - 10 minutes.

After the long wait, you'll see the dashboard

Click on the Network Access tab under the Security group in the sidebar. And then click the ADD IP ADDRESS button.
Click on the button saying Allow access from anywhere and then click on Confirm which will essentially finish this section.

Which means - we finally have everything setup and now we can get started with actually using the database in our program!!!


Setup (inside the code)

Head back to the MongoDB Dashboard's Clusters tab under the Atlas group. And now click on the CONNECT button.
It will ask you to choose a connection method, for which, you need to choose Connect Your Application. In the next, page you'll see two dropdowns: -

  1. Driver - Python
  2. Version - <Whichever version of python you're using> (repl.it uses latest anyways)

It will now show you your connection string. This is a very important thing so be sure to keep it safe.

It is usually of the format -

mongodb+srv://<username>:<password>@<cluster>.mongodb.net/test?retryWrites=true&w=majority

Now come back to repl.it. In the python file where you want to perform DB operations, type the following code

import pymongo

# Define a new client.
client = pymongo.MongoClient("<connection string here>")

# Get the database (database name by default is "test")
db = client.db_name # OR db = client.test

Now that we have our DB, next thing we need to get is a collection. A collection of documents in mongodb can be thought of as a list of dictionaries in python.
By default, a DB doesn't have any collections. To see a list of collections that the database has, the following line of code can be written: -

print(db.list_collection_names())

This will print a list of names (as strings) of all the existing collections in the DB.
Since, there was no collection by default, we'll create one for us! Creating a new collection is very easy itself. All you need to do is

db.create_collection("collection_name")

NOTE: Try to avoid having spaces in collection names, use underscores( _ ) instead.

And now we have a collection where we can store documents which hold our data!

You can also delete an entire collection by using drop_collection instead of create_collection.


Inserting data into a collection

Now, we get to the serious part i.e. inserting data into a collection. For which we have a few choices.

  1. Insert One
    Here, we just insert a single document in the collection.

    NOTE: Python dictionaries are also treated as documents by pymongo.

    Example: -

    # Getting the collection
    my_collection = db.collection_name
    
    # Inserting a sample document.
    my_collection.insert_one({"test": "document", "hello": "world"})

    And that's it! We've successfully inserted a document to our DB! Now let's have a look at the second possibility.

  2. Insert Many
    This is to insert multiple documents to the collection ( Basically a list of dictionaries instead of a single dictionary ).

    Example: -

    # Get the collection
    my_collection = db.collection_name
    
    # Define a list to insert
    documents = [
    {"test": "document", "n": 1},
    {"hello": "world", "n": 2}
    ]
    
    # Insert multiple documents
    my_collection.insert_many(documents)

    And that's how you insert multiple documents at once. It will not create 1 new document and insert the list as it's value, instead, it will create new documents for each of the dicts inside that list!
    And those were the two ways of inserting data.


Searching for documents in a collection

Now that we've inserted some stuff to a collection, we can fetch and view it whenever we want. This can also be done in two ways: -

  1. Find One
    In this method, pymongo will return the first element it finds satisfying the query. The general syntax for this is: -

    db.collection_name.find_one(query)

    Example: -

    # Get the collection
    my_collection = db.collection_name
    
    # Find the document
    x = my_collection.find_one({"field": "value"})

    This example will search for a document in the collection collection_name whose field field's value is value.

  2. Find
    In this method, pymongo will return all the elements it finds satisfying the query. The general syntax for this is: -

    db.collection_name.find(query)

    Example: -

    # Get the collection
    my_collection = db.collection_name
    
    # Find the document
    x = my_collection.find({"field": "value"})

    This example will search for all documents in the collection collection_name whose field field's value is value and return a object of the Cursor class which can be treated as a list of the resulting documents.

And these were the two ways to search for documents that are there in a collection.


Deleting documents

And now, we have some documents that we can get rid of (delete them of course). Deleting can also be done in two ways: -

  1. Delete One

    For deleting just one document. Pretty similar to the find_one function, we need to provide a query object and then it will delete the very first item it finds that satisfies the query.

    Example: -

    # Get the collection
    my_collection = db.collection_name
    
    # Delete the document
    my_collection.delete_one({"field": "value"})

    This will delete the first document it finds which has a field named field whose value is set to value.

  2. Delete Many

    For deleting all similar documents. Pretty similar to the find function, we need to provide a query object and then it will delete the very first item it finds that satisfies the query.

    Example: -

    # Get the collection
    my_collection = db.collection_name
    
    # Delete the document
    my_collection.delete_many({"field": "value"})

    This will delete all the documents it finds which has a field named field whose value is set to value.


Replacing existing documents

Since we have some documents in our DB, sometimes we might need to delete one and add another document simultaneously. Instead of first deleting and then inserting a new document, we can simply use the replace function. This can be achieved by using the replace_one method of the collections which takes two parameters - the query which will allow it to find the existing document to be replaced and the replacement which will be inserted.

Example: -

# Get the collection
my_collection = db.collection_name

# Replace a document
my_collection.replace_one(
    {"some_field": "some_value"},
    {"field_one": "value_one", "field_two": "value_two"}
)

This will first search for a document that has a field named some_field whose value is set to some_value and then replace it with the new document we provided. If it is unable to find the document to be replaced, it will throw an error. The error can be prevented if we pass the optional upsert parameter and set it's value to True as given below: -

my_collection.replace_one(query, new_document, upsert=True)

This will simply insert the new_document if the query did not return any results instead of throwing an error.


Some resources to learn more


The End

If you have any questions, let me know via the comments. If you found this tutorial to be helpful, please upvote.
Thanks for reading it to the end!

You are viewing a single comment. View All
PYer (2511)

Thank you sooooo much. I have been trying to figure out how to get it to work now. Thanks!