MongoDB CRUD with Python

In my last post, we looked at how to combat Impostor Syndrome and came to the conclusion that deepening our knowledge on a given topic helps to sharpen of knowledge sword to fight our feelings of self doubt. If you missed that post, take a look here. This week let’s take a look at using the NoSQL database MongoDB with Python. In a related post, I’ll show how to implement them into web frameworks.

Planning

We will start with an exploration of CRUD operations in MongoDB using Python and PyMongo. Next, we’ll have a look at the Python web framework Bottle and then use Bottle with MongoDB to generate a simple web site. Then we’ll spend a bit of time using MongoDB with the Django web framework to generate a slightly more involved site than our Bottle version.

In this post, we kick off this series with a simple discussion of how to do some simple CRUD operations in Python similar to my post for doing CRUD operations in Java. Not familiar with CRUD? It stands for Create, Read, Update, and Delete. Those are the basic operations one will find themselves doing with any database, not just MongoDB.

In December of 2016, MongoDB released a new version of their database, 3.4, and I’m using 3.4.1 for this series of posts. While we aren’t going to go into any release specific features this week, we might in future posts, so I would encourage you to utilize the latest version. We will be touching on some of the features in the latest version of Python, 3.6, later on as well so make sure you are utilizing that version as well.

Python Driver – PyMongo

Okay, let’s concentrate on some basic implementations and general syntax of the Python driver for MongoDB as well as some basic database documents. Documents, by the way, are similar to a SQL database record.

The project code for this post will be available on GitHub and a link will be included at the end of the post. Let’s start off with our requirements.txt file to list out our project dependencies:

pymongo=='3.4.0'

With that in place we can do pip install -r requirements.txt to install the required version of the libraries for our project.

Another thing we will need for our project to work is a running instance of a MongoDB server. I am using version 3.4.1 and will assume that you have MongoDB 3.4 downloaded, installed, and running on the default address of localhost and port 27017. Further, since we are just sharpening our sword of knowledge here and don’t need MongoDB server authentication, let’s assume that feature is turned off or not implemented at all on our server.

Great, now that our environment is setup let’s enter into the planning stages a bit. What do we need to do?

  • Connect to the database
  • Insert an individual document into the database (Create)
  • Find and print a document in the database (Read)
  • Update the document in the database
  • Finally, Delete the document from the database

Let’s get started!

Database Connection

To setup our connection we will need to import MongoClient from pymongo, set the host and port we intend on using, and then select the database and collection we need. Remember that in MongoDB a collection is similar to a table in the SQL world in that it is a namespace for our data.

from pymongo import MongoClient

HOST = 'localhost'
PORT = 27017

client = MongoClient(HOST, PORT)
database = client.cookbook
recipes = database.recipes

Great, we now have a connection to the MongoDB database cookbook and the collection recipes. But wait, we didn’t ask to specifically create either of those, nor did we put anything into the collection yet. That’s correct, MongoDB generates that for us and waits for us to start using cookbook.recipes. Now, we need to add a basic recipe to our collection so that we have something with which to work.

We’ll define a single recipe document and insert it into our collection using the insert_one() method. Here‘s a link to further documentation on PyMongo’s collection level operations.

recipe = {'title': 'chocolate milk',
          'description': 'Yummy drink',
          'ingredients': [
              {'name': 'milk', 'quantity': 8, 'unit of measure': 'ounce'},
              {'name': 'chocolate syrup', 'quantity': 2, 'unit of measure': 'ounce'}
          ],
          'yield': {'quantity': 1, 'unit': 'glass'},
          'prep time': 0,
          'cook time': 0,
          'author': 'Biff Tannen',
          'uploaded_by': 'kenwalger',
          }

recipes.insert_one(recipe)

That’s all there is to it. We have just added a Chocolate Milk recipe to our collection. We just call the insert_one() method on our collection and pass in our document as an argument.

Reading the documents

Getting a document back out is pretty straight forward as well, we can simply use the find_one() function, which is a collection level operator from the PyMongo driver. It will, as the name would indicate, find a single document in the collection. This can be very useful to get an idea about the schema of the collection. Since we currently only have one record in our database, it should be perfect for getting our information out. I’m also going to include a new import pprint so that when we print out our data in a pretty fashion.

import pprint

print("\nPretty Printed document: \n")
pprint.pprint(recipes.find_one())

Our output should be as follows, with the _id value being different on your system:

Pretty Print: 

{'_id': ObjectId('588541a0146bde28a08217d4'),
 'author': 'Biff Tannen',
 'cook time': 0,
 'description': 'Yummy drink',
 'ingredients': [{'name': 'milk', 'quantity': 8, 'unit of measure': 'ounce'},
                 {'name': 'chocolate syrup',
                  'quantity': 2,
                  'unit of measure': 'ounce'}],
 'prep time': 0,
 'title': 'chocolate milk',
 'uploaded_by': 'kenwalger',
 'yield': {'quantity': 1, 'unit': 'glass'}}

Cool! We have seen how to do two of the four CRUD tasks already! And through the pymongo driver, it has been relatively straightforward to do so. We have a basic recipe for chocolate milk in our cookbook.

Update and Delete

But wait, we incorrectly attributed the recipe to Biff Tannen. As any fan of the Back to the Future movie series knows, chocolate milk is George McFly’s drink, not his arch enemy Biff’s. Let’s Update our database with that change using the update_one() method with the $set operator and print out the new record to make sure it accomplished what we wanted.

recipes.update_one({'title': 'chocolate milk'},
                   {'$set': {'author': 'George McFly'}
                    }
                   )
print("\nShould be George McFly: ")
pprint.pprint(recipes.find_one({'author': 'George McFly'}))

Great, and our output should now reflect that change and we can tell it is the same record because the collection unique _id value is still the same.

Should be George McFly: 
{'_id': ObjectId('588541a0146bde28a08217d4'),
 'author': 'George McFly',
 'cook time': 0,
 'description': 'Yummy drink',
 'ingredients': [{'name': 'milk', 'quantity': 8, 'unit of measure': 'ounce'},
                 {'name': 'chocolate syrup',
                  'quantity': 2,
                  'unit of measure': 'ounce'}],
 'prep time': 0,
 'title': 'chocolate milk',
 'uploaded_by': 'kenwalger',
 'yield': {'quantity': 1, 'unit': 'glass'}}

We have one task left, Delete. Which uses the delete_one() method. We need to pass in an argument for which document to delete which, given that our current collection only has one recipe in there we could use a variety of things to select it. Traditionally we would want to utilize the unique identifier _id to ensure that we are only deleting the document that we want.

recipes.delete_one('_id': ObjectId('588541a0146bde28a08217d4'))

Is all we need to do and our collection is empty again.

CRUD Wrap Up

In summary, we learned how to Create, Read, Update, and Delete documents in MongoDB in Python using the PyMongo driver. There is much improvement on the code to be made though by refactoring things into functions, error handling, etc. As mentioned at the top of this post, the next post will be on Bottle specifically to get up and running with that framework. After that, we will integrate MongoDB with Bottle at which time we will see some best practices for error handling and making these CRUD steps into functions.

Please let me know what you think in the comments below or reach out on Twitter where I am @kenwalger.

The code for this post can be seen in the GitHub repository here.

Happy coding!

Facebooktwitterredditlinkedinmail

Combating Impostor Syndrome

Impostor Syndrome, or feelings of self-doubt, is a common occurrence in just about every field and career. Software developers might experience it more frequently due to the constant and rapid changes in technology we are faced with, but we are not alone. Doctors, professors, attorneys, actors, authors, movie producers, and many, many more all at one time or another seem to suffer from self-doubt or Impostor Syndrome.

Tom Hanks has Impostor Syndrome?

Tom Hanks, for example, has had a rather impressive acting career by most standards. With movies like Apollo 13, Sleepless in Seattle, and Saving Private Ryan. Five Academy Award nominations with two wins, eight Golden Globe Award nominations with four wins, in fact, he has accumulated 17 major award wins in his career. Over a 50% “win” ratio. I’m not an expert on acting award ratios, but winning more often than losing has to be a good standard.

Certainly, someone who has the acting resume, awards, and career that Mr. Hanks has must not experience Impostor Syndrome and self-doubt, right? Here’s a recent quote of his:

“I still feel sometimes that I’d like to be as good as so-and-so actor. I see some other actors’ work, and I think I’ll never get there. I wish I could.” – Tom Hanks

Clearly, if Mr. Hanks has doubts about his acting abilities in comparison to others, we shouldn’t be shocked when we encounter feelings of being an impostor in our own roles.

Can you relate?

I would venture to guess that we have all had people we look up to in our jobs or industry. People that we believe have knowledge, experience, or talent that we will never be able to achieve. Remember back to when you were first starting out in your current career. I know, for some that is much more recent than others, but think back. Recall a time when you were watching your mentor fly through some project code or teach a difficult subject. Did you feel like “Wow, Trina is amazing! I’ll never be as good as her?” Or maybe you were struggling on a project and after hours of trying to figure out a solution Travis walks by, points out your mistake and just walks away shaking his head after spending seconds on the project?

Both scenarios start that feeling of self-doubt. That feeling of being an impostor in our chosen field. That feeling that someone is going to find out that we are frauds and should be doing something different.

Fighting a dragon

But how can we combat that feeling? How can we prevent it from taking over our thoughts and stifling our work flow and creativity?

I like to think of combating impostor syndrome in a metaphorical way of a knight fighting a dragon. Prior to going off to battle the dragon that is Impostor Syndrome we need to get our mental armor on. As with a suit of armor and tools that a knight would wear and use, we need to put on mental armor when we start on a new task or project to prevent our struggles with learning from overwhelming us. Things like a helmet, breast plate, leggings, boots, sword, and shield are all needed to fight our evil dragon.

Our Armor

Helmet

I think of the helmet as protecting bad thoughts from entering and good thoughts from exiting my brain. We all have knowledge and skills to bring to the table, it is often a matter of being humble enough to admit that we do know things and restructure what we think others know.As we can see, we do know things about our tasks, and how to implement them. Keeping our thoughts protected can help us remember that.

Breast Plate

Our breast plate (which will for the sake of argument include arm and hand protection) I think of as protecting us from negative feedback. Let’s face it, not everyone is as nice or tolerant of others as we would like. Often when we are working with someone who we highly respect in our careers, they can do or say something that enhances our impostor feelings. “How long have you been staring at that code, Joe? Of course, it isn’t working, that line isn’t indented correctly. Duh!” Huge blow when that happens to us, right? Keeping a breast plate on us to allow those comments to not have as large of an impact is important to prevent the impostor dragon from causing overwhelming damage.

Leggings

The leggings are important as well. Obviously without our legs walking, running, and moving about to engage in combat would be a challenge. To combat our dragon we need to engage in community, realize that we are not alone in feeling like an impostor. Everyone has something to contribute to a project, and to the battle to fight our dragon.

Boots

Boots, or in the language of armor sabatons, protect our feet, which we need to make steps along our journey. Keep a journal of the steps we take as we walk along our journey. Write down successes and, yes, failures. It provides retrospective insights about both conditions and revisiting them allows us to remember both equally as well.

The shield

Thus far everything has been something we would actually wear. A shield is something we carry with us, right? In this metaphor, a shield represents an idea to shield us. That idea is that knowing that no one is perfect. No one. There is no perfect actor, doctor, author, professors, and there certainly isn’t a perfect developer. Knowing that even the most seasoned developer on your team likely suffers from impostor syndrome from time to time as well will help shield you from your own thoughts of being an impostor and will help fight off that evil dragon.

Armor recap

These are the basics of our armor: a helmet to protect our thoughts, a breast plate to protect us from negative feedback, leggings to remember that we are not alone out there, boots as a reminder to keep a log of our journey, and a shield to protect against the idea that the perfect developer exists. Now we get to our weapon, the sword. Imagine going into battle against a dragon without an offense?

Our sword is used to represent knowledge. That is what ultimately will slay our dragon, right? When we feel confident in a certain task, we no longer feel like an impostor in that one area. Remember that even the most experienced people continue to fight and search for answers and gain knowledge in the process.

With our armor, shield, and sword set and ready to fight our dragon, we should likely have a battle plan. Fortunately, if we keep our protection in place it simplifies our battle quite considerably.

Wrapping it up

For new developers (and by “new” I am talking not only about new to the industry, but also new to a given subject or technology), remember that everyone in a development role is in a constant learning state. Find a good mentor as part of your game plan, it will help in sharpening your sword (knowledge) to fight our dragon.

If you are a seasoned developer, you can help in our battle plan as well. While you might not be struggling with writing a “Hello world” app in Java 7, remember that there are things to learn in Java 9 that might cause you to resharpen your sword. Use your experience to mentor junior developers and put yourself in their shoes. Remember what it was like to push your first repo to GitHub, to make your first pull request to an Open Source library, or your first venture into using a new Python library.

Let’s all try to keep our helmet, breast plate, leggings, boots, and shields in tact to protect ourselves during those times in which we are facing our dragon named Impostor Syndrome. And keep up on our learning and mentoring as well, to keep our sword sharp to battle and defeat the dragon.

Facebooktwitterredditlinkedinmail