{"id":608,"date":"2017-10-03T05:53:38","date_gmt":"2017-10-03T12:53:38","guid":{"rendered":"https:\/\/www.kenwalger.com\/blog\/?p=608"},"modified":"2017-12-01T07:58:32","modified_gmt":"2017-12-01T15:58:32","slug":"storing-documents-mongodb-database","status":"publish","type":"post","link":"https:\/\/www.kenwalger.com\/blog\/nosql\/mongodb\/storing-documents-mongodb-database\/","title":{"rendered":"Storing Documents in a MongoDB Database?"},"content":{"rendered":"

MongoDB<\/a>\u00a0is considered a document database. As happens from time to time in the world of technology, the same word is used for different concepts. In this case, the word “document” can sometimes cause confusion. Is\u00a0storing documents the job of the database?<\/p>\n

For many of us, when we hear the word “document” we think of a Microsoft Word<\/a>\"\" document. Some may think of Adobe Acrobat<\/a>\"\"‘s Portable Document Format files. Or any other common “document” concepts. In MongoDB and, in fact, many other database models the term\u00a0document<\/a> has a different meaning. I wrote a post about the document model<\/a> as it pertains specifically to MongoDB and would recommend having a look for further information. In a nutshell, however, in MongoDB, a document is an individual record of information.<\/p>\n

What if we actually want to store “documents” in a database? Is there a best practice for storing documents with MongoDB? What are some options to consider?<\/p>\n

MongoDB GridFS<\/h3>\n

GridFS<\/a> allows for the storage of more data in a single MongoDB document than is allowed by the BSON specification<\/a> of 16MB. This allows for the database record to be split into separate files on disk. This can allow for some pretty specific benefits in addition to those laid out in the GridFS documentation. For example, with replication<\/a>, one could keep files deployed across multiple facilities and utilize MongoDB’s synchronization\u00a0features. However, many file systems account for this as well on their own.<\/p>\n

Storing documents with GridFS is a topic in an of itself and beyond to scope of this blog post. It’s definitely on the list of things to cover, though.<\/p>\n

Another Option for Storing Documents<\/h3>\n

Let me start here by saying that I am a bit biased in this regard. I have never been a huge fan of storing “documents”, images, videos, etc.\u00a0inside<\/em> a database. It has always been my preference to store files on a file server, network attached storage<\/a>\"\" system, or in the cloud somewhere like Amazon S3<\/a>\"\".<\/p>\n

How then does one combine these two concepts? Store the files on disk outside the database, but utilize the power of the database as well? Well, you use the MongoDB document model and flexible schema to handle the metadata of the file, along with a link to the location of the file. Something along the lines of:<\/p>\n

{\n    _id : ,\n    file_title : \"<title>\",\n    file_URL : \"<S3_URL>\",\n    uploaded_by : \"<user_name>\",\n    created_at : <Timestamp>,\n    updated_at : <Timestamp>,\n    updated_by : \"<user>\",\n    department : \"<department_name>\",\n    tags : [\n        { name : \"<tag1>\" },\n        { name : \"<tag2>\" }\n    ],\n    version : <Timestamp>\n}\n<\/pre>\n
This sort of design allows for some nice performance and query possibilities. If, for example, we want to search for “all documents tagged with the Python topic uploaded by Ken between 01 Jan 2016 and 01 Oct 2017” we can query for that. We also get to utilize the file system itself which is often better optimized for file storage reads and writes.<\/p>\n
Wrap Up<\/h3>\n
In this short post, I hope to have clarified the concept of a document that is stored in a MongoDB database, like a Word or PDF file. Versus the concept of a document that is<\/em> a MongoDB database record. While MongoDB can be utilized for file storage, in my opinion, file storage should be left to, well, file storage systems.<\/p>\n
Design your database schema to maintain the metadata and links for your documents and take advantage of MongoDB’s rich query language for finding your information.<\/p>\n
\n
Follow me on Twitter @kenwalger<\/a> to get the latest updates on my postings.<\/p>\n
There are a few MongoDB specific terms in this post. I created a MongoDB Dictionary<\/a> skill for the Amazon Echo<\/a>\u00a0line of products. Check it out and you can say “Alexa, ask MongoDB for the definition of a\u00a0document?” and get a helpful response.<\/p>\n\"Facebook\"<\/a>\"twitter\"<\/a>\"reddit\"<\/a>\"linkedin\"<\/a>\"mail\"<\/a>","protected":false},"excerpt":{"rendered":"
MongoDB\u00a0is considered a document database. As happens from time to time in the world of technology, the same word is used for different concepts. In this case, the word “document” can sometimes cause confusion. Is\u00a0storing documents the job of the database? For many of us, when we hear the word “document” we think of a … Continue reading “Storing Documents in a MongoDB Database?”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":616,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"pmpro_default_level":"","_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[4],"tags":[980,979],"yst_prominent_words":[982,104,983,298,286,538,767,984,972,975,971,87,411,976,722,575,945,973,968,969],"class_list":["post-608","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-mongodb","tag-document-storage","tag-gridfs","pmpro-has-access"],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/www.kenwalger.com\/blog\/wp-content\/uploads\/2017\/10\/wp_feature_store_documents-e1506995740114.png?fit=125%2C125&ssl=1","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p8lx70-9O","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/posts\/608","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/comments?post=608"}],"version-history":[{"count":8,"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/posts\/608\/revisions"}],"predecessor-version":[{"id":620,"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/posts\/608\/revisions\/620"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/media\/616"}],"wp:attachment":[{"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/media?parent=608"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/categories?post=608"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/tags?post=608"},{"taxonomy":"yst_prominent_words","embeddable":true,"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/yst_prominent_words?post=608"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}