Skip to main content

MongoDB to BigQuery Template

Additional Configurations#

  • Ignore Fields: Fields will be ignored when extracting data from MongoDB. (Separated by comma ,)

  • Custom Schema: By default, table schema will be auto generate if custom schema is not defined. On the other hand, if custom schema is provided, it will extract only fields that are defined in schema. If that the case then you will need to upload schema file to GCS, and provide GCS URI.

    Example

    gs://bucket/folder/schema_file.json
note

You have to provide a valid BigQuery Schema Format, see here

  • Flatten Data: Convert all nested fields to separated columns with format {parent}_{child}

    Example

      {      "field1": {          "field2": "value"      },      "field3": {          "field4": {              "field5": "value2"          }      }  }

    will be converted to

      {      "field1_field2": "value,      "field3_field4_field5": "value2",  }