Build and run a custom image

• Use a prebuilt image
• Build a local image
  • Clone the stable branch only
  • Sync code to the stable branch
  • Build the repo locally
• Run the services container locally
• Troubleshooting

Use a prebuilt image

While you are just testing out data changes, you don’t need to build the website, but can just use a prebuilt Data Commons image.

Data Commons provides two prebuilt images in the Google Artifact Registry that you can download to run in a Docker container:

• gcr.io/datcom-ci/datacommons-data:stable and gcr.io/datcom-ci/datacommons-services:stable. These are tested, stable versions but may be several weeks old.
• gcr.io/datcom-ci/datacommons-data:latest and gcr.io/datcom-ci/datacommons-services:latest. These are the latest versions built from head.

If you want to pick up the latest prebuilt version, do the following:

1. From the root directory (e.g. website), run the following command:

   docker pull gcr.io/datcom-ci/datacommons-services:latest
   docker pull gcr.io/datcom-ci/datacommons-data:latest
   

2. Rerun the data container, specifying that repo specifying that repo as the argument to the docker run command:

   docker run \
   --env-file $PWD/custom_dc/env.list \
   -v INPUT_DIRECTORY:INPUT_DIRECTORY \
   -v OUTPUT_DIRECTORY:OUTPUT_DIRECTORY \
   gcr.io/datcom-ci/datacommons-data:latest
   

3. Restart the services container:

   docker run -it \
   -p 8080:8080 \
   -e DEBUG=true \
   --env-file $PWD/custom_dc/env.list \
   -v INPUT_DIRECTORY:INPUT_DIRECTORY \
   -v OUTPUT_DIRECTORY:OUTPUT_DIRECTORY \
   gcr.io/datcom-ci/datacommons-services:latest
   

Build a local image

You will need to build a local image in any of the following cases:

• You are making substantive changes to the website UI
• You are ready to deploy your custom site to GCP

Building from the master branch includes the very latest changes in Github, that may not have been tested. Instead, we recommend that you use the tested “stable” branch equivalent of the stable Docker image. This branch is customdc_stable, and is available at https://github.com/datacommonsorg/website/tree/customdc_stable.

Note: If you are working on a large-scale customization, we recommend that you use a version control system to manage your code. We provide some procedures for Github.

Clone the stable branch only

Use this procedure if you are not using Github, or if you are using Github and want to create a new source directory and start from scratch.

1. Run the following command:

   git clone https://github.com/datacommonsorg/website --branch customdc_stable --single-branch  [DIRECTORY]
   

   This creates a new local branch called customdc_stable set to track the Data Commons repo branch.

2. To verify, run:

   cd website | cd DIRECTORY
   git branch -vv
   

   You should see output like the following:

   * customdc_stable 83732891 [origin/customdc_stable] 2024-11-06 Custom DC stable release (#4710)
   

   Rather than developing on this default branch, we recommend that you create another branch.

Sync code to the stable branch

The following procedure uses Github. If you are using another version control system, use the appropriate methods for updating submodules and syncing.

1. Switch to the directory where you have cloned the Data Commons doe:

   cd website | cd DIRECTORY
   

2. Update files:

   git pull origin customdc_stable
   

   Note that origin here refers to the source datacommonsorg/website repo. You may be using another remote name to point to that repo.

   You should see output like the following:

   From https://github.com/datacommonsorg/website
   * branch              customdc_stable -> FETCH_HEAD
   Already up to date.
   

3. Create and update the necessary submodules:

   git submodule update --init --recursive
   

   You should see output like the following:

   Submodule 'import' (https://github.com/datacommonsorg/import.git) registered for path 'import'
   Submodule 'mixer' (https://github.com/datacommonsorg/mixer.git) registered for path 'mixer'
   Submodule path 'import': checked out '7d197583b6ad0dfe0568532f919482527c004a8e'
   Submodule path 'mixer': checked out '478cd499d4841a14efaf96ccf71bd36b74604486'
   

Build the repo locally

Run the following command to build the repo:

docker build --tag IMAGE_NAME:IMAGE_TAG \
-f build/cdc_services/Dockerfile .

• The image name is a meaningful name, such as datacommons-services.
• The image tag is a meaningful description of the version you are building, such as latest.

It will take several minutes to build.

To run the container with the local SQLite database, start the Docker container as described below.

To upload and deploy the container to the Cloud, see Deploy services to Google Cloud for procedures.

Run the services container locally

Start the services using the locally built repo. If you have made changes to any of the UI components (or directories), be sure to map the custom directories (or alternative directories) to the Docker workspace directory.

docker run -it \
--env-file $PWD/custom_dc/env.list \
-p 8080:8080 \
-e DEBUG=true \
-v INPUT_DIRECTORY:INPUT_DIRECTORY \
-v OUTPUT_DIRECTORY:OUTPUT_DIRECTORY \
[-v $PWD/server/templates/custom_dc/custom:/workspace/server/templates/custom_dc/custom \]
[-v $PWD/static/custom_dc/custom:/workspace/static/custom_dc/custom \]
IMAGE_NAME:IMAGE_TAG

Troubleshooting

Having trouble? Visit our Troubleshooting Guide for detailed solutions to common problems.

Page last updated: December 17, 2024 • Send feedback about this page