{"id":704,"date":"2017-11-28T12:58:30","date_gmt":"2017-11-28T20:58:30","guid":{"rendered":"https:\/\/www.kenwalger.com\/blog\/?p=704"},"modified":"2017-11-28T12:58:30","modified_gmt":"2017-11-28T20:58:30","slug":"schema-validation-mongodb-3-6","status":"publish","type":"post","link":"https:\/\/www.kenwalger.com\/blog\/nosql\/mongodb\/schema-validation-mongodb-3-6\/","title":{"rendered":"Schema Validation in MongoDB 3.6"},"content":{"rendered":"

MongoDB 3.6 brings lots of great new features with the new release. I’ve already covered Change Streams<\/a> and Retryable Writes<\/a> in previous posts. This post will cover a feature which expands upon the document validation<\/a> feature from MongoDB 3.2, schema validation. Schema validation allows for teams to define a prescribed document structure for each collection.<\/p>\n

There are times when enforcing strict data structures and content are required, even with the flexible schema and document data model<\/a> that MongoDB<\/a> provides. Schema validation allows for the ability to define a prescribed document structure for each collection. If one tries to insert or update a document which does\u00a0not<\/em> conform to the applied structure the operation can be rejected. The rules for document structure are based on the J<\/a>SON schema draft specification<\/a>.<\/p>\n

Schema Validation<\/h3>\n

Let’s take a quick look at how this works in action before diving into a discussion of feature benefits.<\/p>\n

Imagine a collection of food recipes. Each recipe will have a recipe name, number of servings, cooking method, ingredients, and list of instructions. For the\u00a0ingredients, we\u00a0want to enforce that there is a numerical quantity, a measure of quantity, the ingredient name, and an optional value for any prep work on the ingredient, such as “peeled” or “brunoise<\/a>\"\"“.\u00a0 For our example here my options are not all-inclusive for options but simply serve as examples. I hope my former culinary colleagues forgive me.<\/p>\n

We’ll begin with creating a recipes<\/code> collection and assigning the schema validation rules with the validator<\/code> option and the new $jsonSchema<\/a> operator.<\/p>\n

db.createCollection( \"recipes\",\n{\n  validator: \n    {\n      $jsonSchema:\n        {\n          bsonType: \"object\",\n          required: [\"name\", \"servings\", \"ingredients\"],\n          additionalProperties: false,\n          properties:\n            {\n              _id: {},\n              name: {\n                bsonType: \"string\",\n                description: \"'name' is required and is a string\"\n                    },\n              servings: {\n                bsonType: [\"int\", \"double\"],\n                minimum: 0,\n                description: \"'servings' is required and must be an integer greater than zero.\"\n                    },\n              cooking_method: {\n                 enum: [\"broil\", \"grill\", \"roast\", \"bake\", \"saute\", \"pan-fry\", \"deep-fry\", \"poach\", \"simmer\", \"boil\", \"steam\", \"braise\", \"stew\"],\n              description: \"'cooking_method' is optional but, if used, must be one of the listed options.\"\n                    },\n              ingredients: \n              {\n                bsonType: [\"array\"],\n                minItems: 1,\n                maxItems: 50,\n                items: {\n                  bsonType: [\"object\"],     \n                  required: [\"quantity\", \"measure\", \"ingredient\"],\n                  additionalProperties: false,\n                  description: \"'ingredients' must contain the stated fields.\",\n                  properties: \n                  {\n                    quantity: {\n                      bsonType: [\"double\", \"decimal\"],\n                      description: \"'quantity' is required and is of double or decimal type\"\n                            },\n                    measure: {\n                      enum: [\"tsp\", \"Tbsp\", \"cup\", \"ounce\", \"pound\",  \"each\"],\n                      description: \"'measure' is required and can only be one of the given enum values\"\n                            },\n                    ingredient: {\n                      bsonType: \"string\",\n                      description: \"'ingredient' is required and is a string\"\n                            },\n                    format: {\n                      bsonType: \"string\",\n                      description: \"'format' is an optional field of type string\"\n                            }\n                  }\n              }\n           }\n        }\n     }\n   }\n})<\/pre>\n
Our validator<\/code> can include many more rules<\/a> that are beyond the scope of this introduction to schema validation. However, we can see how powerful this feature is in this small example. Let’s look at some example documents to insert as well to see what would happen if we try to insert<\/a> a document<\/a> into the collection<\/a>.<\/p>\n
Sample Inserts<\/h6>\n
db.recipes.insertOne({name: \"Chocolate Sponge Cake Filling\", \nservings: 4, \ningredients: [{quantity: 7, measure: \"ounce\", ingredient: \"bittersweet chocolate\", format: \"chopped\"}, \n{quantity: 2, measure: \"cup\", ingredient: \"heavy cream\"}\n]})<\/pre>\n
This insert works since it covers all of the required fields in their proper format. It also doesn’t include any invalid extra fields which would be prohibited since we have\u00a0additionalProperties: false<\/code> set. If we were to try to insert the following document, however, we would get an error.<\/p>\n
db.recipes.insertOne({name: \"Chocolate Sponge Cake Filling\", \nservings: 4, \ningredients: [{quantity: 7, measure: \"ounce\", ingredient: \"bittersweet chocolate\", format: \"chopped\"}, \n{quantity: 2, measure: \"cup\", ingredient: \"heavy cream\"}],\ndirections: \"Boil cream and pour over chocolate. Stir until chocolate is melted.\"\n})<\/pre>\n
Since we added in a directions<\/code> field into our recipe document, the insert will fail with\u00a0additionalProperties: false<\/code>\u00a0set. I should add in an optional directions<\/code> field into the schema validation to allow for that as directions are indeed important.<\/p>\n
Schema Validation Benefits<\/h3>\n
Even with these basic examples, I think it is clear that schema validation is a great new enhancement. The flexibility of a dynamic schema in MongoDB can now easily be paired with data governance controls over an entire collection. While there are lots of practical benefits from this, here are some specific benefits.<\/p>\n
    \n
  1. Application logic simplification. With a strictly defined definition of what a collection looks like, there isn’t a need for the application to handle the guarantees and associated errors.<\/li>\n
  2. Control of data. Client applications must adhere to the rules set forth by the collection. No need to worry about data being updated or inserted that has incorrect field names or data types.<\/li>\n
  3. Governmental compliance. There are applications and data models in a variety of industries and locales that require data to be stored in specific formats. For example, the EU General Data Protection Regulation<\/a>\"\" in the European Union.<\/li>\n<\/ol>\n
    Wrap Up<\/h3>\n
    With MongoDB 3.6 schema validation, administrators have tunable controls over the database. Documents that don’t conform to a prescribed set of conditions can be rejected or still written and a message logged about the action. It also brings with it the ability to query based on the schema definition to allow,\u00a0for example, a search on all documents that\u00a0don’t<\/em> conform to the schema.<\/p>\n
    Schema validation is another exciting feature in the 3.6 release of MongoDB. I would urge you to download<\/a> it and try it out. I’d bet you’ll be as excited about it as I am.<\/p>\n
    \n
    Follow me on Twitter @kenwalger<\/a> to get the latest updates on my postings. Or better yet, sign up for the email list to get updates in your mailbox!<\/p>\n
    There are a few MongoDB specific terms in this post. I created a MongoDB Dictionary<\/a> skill for the Amazon Echo<\/a>\u00a0line of products. Check it out and you can say “Alexa, ask MongoDB for the definition of a\u00a0document?” and get a helpful response.\u00a0I also created a culinary skill<\/a> for the Echo if you’d like to update your culinary knowledge as well.<\/p>\n\"Facebook\"<\/a>\"twitter\"<\/a>\"reddit\"<\/a>\"linkedin\"<\/a>\"mail\"<\/a>","protected":false},"excerpt":{"rendered":"
    MongoDB 3.6 brings lots of great new features with the new release. I’ve already covered Change Streams and Retryable Writes in previous posts. This post will cover a feature which expands upon the document validation feature from MongoDB 3.2, schema validation. Schema validation allows for teams to define a prescribed document structure for each collection. … Continue reading “Schema Validation in MongoDB 3.6”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":709,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"pmpro_default_level":"","_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[4],"tags":[1102,1142],"yst_prominent_words":[117,99,1147,1150,1149,298,1152,360,1061,1163,87,1099,1153,1151,782,1143,1144,1154,1141,1145],"class_list":["post-704","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-mongodb","tag-3-6","tag-schema-validation","pmpro-has-access"],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/www.kenwalger.com\/blog\/wp-content\/uploads\/2017\/11\/document_validation_feature-e1511822206981.png?fit=125%2C125&ssl=1","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p8lx70-bm","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/posts\/704","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/comments?post=704"}],"version-history":[{"count":5,"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/posts\/704\/revisions"}],"predecessor-version":[{"id":715,"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/posts\/704\/revisions\/715"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/media\/709"}],"wp:attachment":[{"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/media?parent=704"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/categories?post=704"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/tags?post=704"},{"taxonomy":"yst_prominent_words","embeddable":true,"href":"https:\/\/www.kenwalger.com\/blog\/wp-json\/wp\/v2\/yst_prominent_words?post=704"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}