{"id":554,"date":"2017-08-24T08:31:13","date_gmt":"2017-08-24T15:31:13","guid":{"rendered":"https:\/\/www.kenwalger.com\/blog\/?p=554"},"modified":"2017-08-24T17:33:29","modified_gmt":"2017-08-25T00:33:29","slug":"data-durability-mongodb","status":"publish","type":"post","link":"https:\/\/www.kenwalger.com\/blog\/nosql\/mongodb\/data-durability-mongodb\/","title":{"rendered":"Data Durability in MongoDB"},"content":{"rendered":"

When designing a database we want to make sure the data that we want to be stored actually gets<\/strong> stored. Data durability is a key factor in applications. \u00a0On local servers and test environments, this typically isn’t a huge issue. We can pretty easily tell when and if our environment crashes. What happens though as our system grows? What happens when we move to a distributed environment with many different pieces to our application puzzle?<\/p>\n

In addition to many of the performance<\/a> considerations that MongoDB<\/a> has improved upon in recent versions, data durability is another improvement. It is also a topic from previous versions for which the product took some heat. Let’s take a look at some ways which we can design our applications around the idea of data durability.<\/p>\n

Data Durability<\/h3>\n

There are a couple of different scenarios we need to consider when dealing with data durability, reads and writes. Let’s take a look at these two different considerations. In doing so we’ll see some ways to ensure our system is doing what we intend it to be doing.<\/p>\n

Writes<\/h4>\n

Write operations in MongoDB follow a pretty clear path in MongoDB. At least in theory. From an application, they get sent to the primary<\/a> server, in a replica set<\/a> situation. The data goes into an in-memory store and oplog<\/a>. At this point, the server, by default, sends back an “okey dokey” to the application.<\/p>\n

Notice that I haven’t mentioned anything about writing data to disk yet. It hasn’t happened yet. The primary then writes the data to the journal<\/a> file and then to disk. The secondaries are writing data to the disk during this process as well, at some point after the data has been written to the journal.<\/p>\n

This can be all well and good in many situations as we are talking about small time frames between the application getting an “okay” and the data being persisted to disk. But there is still some latency there. Should something go wrong with the distributed system during that time, extra steps have to be taken to get the data. The application thinks is there, it did get a confirmation of it after all, and what actually took place with the disk writes.<\/p>\n

I’m not going to go into the background of what\u00a0actually<\/em> goes on behind the scenes during an unexpected shutdown or failure. It is a bit beyond the scope of this particular post. I will show how to instruct MongoDB to wait to send our “okey dokey” signal to the application until the data is indeed on disk.<\/p>\n

Write Acknowledgment<\/h6>\n

MongoDB has provided the functionality to set a level of acknowledgment for write operations with the write concern<\/a> options. There are a few different options available for us here.<\/p>\n