Build and run a custom image
Use a prebuilt image
While you are just testing out data changes, you don’t need to build the website, but can just use a prebuilt Data Commons image.
Data Commons provides two prebuilt images in the Google Artifact Registry that you can download to run in a Docker container:
gcr.io/datcom-ci/datacommons-services:stable
. This is a tested, stable version but may be several weeks old.gcr.io/datcom-ci/datacommons-services:latest
. This is the latest version built from head.
If you want to pick up the latest prebuilt version, do the following:
-
From the root directory (e.g.
website
), run the following command:docker pull gcr.io/datcom-ci/datacommons-services:latest
-
Rerun the container, specifying that repo as the argument to the
docker run
command:docker run -it \ -p 8080:8080 \ -e DEBUG=true \ --env-file $PWD/custom_dc/env.list \ -v $PWD/custom_dc/sample:$PWD/custom_dc/sample \ gcr.io/datcom-ci/datacommons-services:latest
Build a local image
You will need to build a local image in any of the following cases:
- You are making substantive changes to the website UI
- You are ready to deploy your custom site to GCP
Rather than building from “head”, that is, the very latest changes in Github, which may not have been tested, we recommend that you use the tested “release” equivalent of the stable Docker image. This release uses the tag customdc_stable
, and is available at https://github.com/datacommonsorg/website/releases/tag/customdc_stable.
Note: If you are working on a large-scale customization, we recommend that you use a version control system to manage your code. We provide procedures for Github, and assume the following:
- You have a Github account and project.
- You have created a fork off the base Data Commons
website
repo (https://github.com/datacommonsorg/website){: target=”_blank”} and a remote that points to it, and that you will push to that fork.
Sync a local workspace to the stable release
If you are using a version control system other than Github, you can download a ZIP or TAR file from https://github.com/datacommonsorg/website/releases/tag/customdc_stable.
In Github, use the following procedure.
-
If you want to reuse the root directory you previously created and cloned, skip to step 3. If you want to create a new source directory and start from scratch, clone the repo up to the stable release tag:
git clone https://github.com/datacommonsorg/website --branch customdc_stable --single-branch [DIRECTORY]
-
Change to the root directory:
cd website | cd DIRECTORY
-
Create a new branch synced to the stable release:
git checkout -b BRANCH_NAME customdc_stable
-
To verify that your local repo is at the same version of the code, run the following command:
git log --oneline --graph
You should see output similar to the following:
* 52635c8 (grafted, HEAD -> branch1, tag: customdc_stable) ... ...
Verify that the last commit in the output matches that listed in https://github.com/datacommonsorg/website/releases/tag/customdc_stable.
-
Press
q
to exit the output log. -
Create and update the necessary submodules:
git submodule foreach git pull origin customdc_stable git submodule update --init --recursive
You should see output like the following:
Submodule 'import' (https://github.com/datacommonsorg/import.git) registered for path 'import' Submodule 'mixer' (https://github.com/datacommonsorg/mixer.git) registered for path 'mixer' Submodule path 'import': checked out '7d197583b6ad0dfe0568532f919482527c004a8e' Submodule path 'mixer': checked out '478cd499d4841a14efaf96ccf71bd36b74604486'
-
Update all other files:
git pull origin customdc_stable
You will likely see the following output:
From https://github.com/datacommonsorg/website * tag customdc_stable -> FETCH_HEAD Already up to date.
Build the repo locally
Run the following command to build the repo:
docker build --tag IMAGE_NAME:IMAGE_TAG \ -f build/cdc_services/Dockerfile .
- The image name is a meaningful name, such as
datacommons-services
. - The image tag is a meaningful description of the version you are building, such as
latest
.
It will take several minutes to build.
To run the container with the local SQLite database, start the Docker container as described below.
To upload and deploy the container to the Cloud, see Deploy services to Google Cloud for procedures.
Run the services container locally
Start the services using the locally built repo. If you have made changes to any of the UI components (or directories), be sure to map the custom
directories (or alternative directories) to the Docker workspace
directory.
docker run -it \ --env-file $PWD/custom_dc/env.list \ -p 8080:8080 \ -e DEBUG=true \ -v OUTPUT_DIRECTORY:OUTPUT_DIRECTORY \ [-v $PWD/server/templates/custom_dc/custom:/workspace/server/templates/custom_dc/custom \] [-v $PWD/static/custom_dc/custom:/workspace/static/custom_dc/custom \] IMAGE_NAME:IMAGE_TAG