Dockerfile update (#1270)

While Docker isn't officially supported by Hometown, leaving the Mastodon 3.5.5 Docker configuration in place with the new 4.0.2 code is a bad idea. At minimum, you'll have a stale Node install that's months behind on security updates. There are some minor tweaks to the default configuration, but they're flagged by comments so they're easy to revert or modify as necessary. # Running Hometown on Docker I'll by typing up my own longer blog post in due time, but there's no harm dropping a cheat sheet here. By following this outline, I was able to upgrade a Hometown 1.0.8 install to 1.1.0 with nothing worse than a minute or two of downtime. My configuration uses the GitHub repository as its source, rather than images drawn from DockerHub. I like to tweak and fiddle with my setup, especially the themes, and I'm happy to sacrifice some disk space for the privilege. ## Installing from Scratch This is by far the easiest approach, you just follow [one of](https://gist.github.com/TrillCyborg/84939cd4013ace9960031b803a0590c4) the [existing guides](https://sleeplessbeastie.eu/2022/05/02/how-to-take-advantage-of-docker-to-install-mastodon/) for running Mastodon via Docker, pause after you've set up `.env.production`, add any Hometown-specific features to it [as per the Wiki](https://github.com/hometown-fork/hometown/wiki), then resume what the guide says to do. If you're enabling ElastiSearch, the second of the two guides has some additional actions you'll need to do, plus be aware of [this bug](https://github.com/mastodon/mastodon/issues/18625) in Mastodon which can quietly block ES from working at all. ## Upgrading from Hometown 1.0.8 Here's how I accomplished this. I committed any leftover changes, then ran these commands from the non-Docker instructions in the root of my local Hometown repository: ``` git remote update git checkout v4.0.2+hometown-1.1.0 ``` This "wiped out" my customizations, but as I committed them all to a branch I can reconstruct them later via diffs. I then ran: ``` sudo docker-compose build ``` to build the new image. The old image will continue running in the background, as per usual. I like adding `2>&1 | less` to the end and mashing `PgDn`, as if a compilation error happens it almost invariably requires scrolling back a few screens to find the issue. If the build succeeded, we're almost clear to start the dangerous portion. If you're running on the cloud, now would be a great time to take a snapshot. Whatever the case, you should back up the existing database. If you haven't changed the defaults from the Dockerfile, then ``` sudo docker exec -it hometown_db_1 pg_dump -U postgres -Fc postgres > hometown.db.dump ``` should do the trick. If you have changed the defaults, you may need to use `sudo docker ps` to figure out the name of the PostgreSQL image to swap in place of "hometown_db_1", then browse through `.env.production` to extract the username to place after `-U` and the database name to place after `-Fc`. The Hometown docs don't say how to restore the database should the process go South, but after reading a manpage or two I think the magic words are roughly ``` sudo docker exec -it hometown_db_1 pg_restore -U postgres --clean --if-exists -d postgres < hometown.db.dump ``` Now we're ready for the scary "you could destroy everything" part. All the earlier commands are trivial to roll back, but after this point any delay could cause data corruption. As per the Hometown docs, run the pre-deployment database migrations. ``` sudo docker-compose run -e SKIP_POST_DEPLOYMENT_MIGRATIONS=true -e RAILS_ENV=production --rm web bundle exec rails db:migrate ``` where `web` is the name of the webserver image in `docker-compose.yml`. The docs state you should precompile all assets next, but I'm 95% sure they were already built when you ran `sudo docker-compose build`. If you're paranoid and want to be absolutely sure precompilation is done, then at this stage run: ``` sudo docker-compose run -e RAILS_ENV=production --rm web bundle exec rails assets:precompile ``` Here, the Hometown docs say you should run the post-deployment migrations. In Docker-ese: ``` sudo docker-compose run -e RAILS_ENV=production --rm web bundle exec rails db:migrate ``` Finally, we need to stop the old images and spin up the new ones. Run: ``` sudo docker-compose up -d ``` and give Docker some time to finish rotating. A quick `sudo docker ps` should confirm the new images are booting up, and in a short while (10-15 seconds for the teeny-tiny instance I manage) you should be back to fully functional.
2023-01-17 15:39:09 -07:00 · 2023-01-17 15:39:09 -07:00 · d7e2a5c6e7
parent 1385bb319e
commit d7e2a5c6e7
2 changed files with 79 additions and 101 deletions
--- a/170
+++ b/170
@ -1,121 +1,99 @@
-FROM ubuntu:20.04 as build-dep
+# syntax=docker/dockerfile:1.4
+# This needs to be bullseye-slim because the Ruby image is built on bullseye-slim
+ARG NODE_VERSION="16.18.1-bullseye-slim"

-# Use bash for the shell
-SHELL ["/bin/bash", "-c"]
-RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
+FROM ghcr.io/moritzheiber/ruby-jemalloc:3.0.4-slim as ruby
+FROM node:${NODE_VERSION} as build

-# Install Node v16 (LTS)
-ENV NODE_VER="16.17.1"
-RUN ARCH= && \
-    dpkgArch="$(dpkg --print-architecture)" && \
-  case "${dpkgArch##*-}" in \
-    amd64) ARCH='x64';; \
-    ppc64el) ARCH='ppc64le';; \
-    s390x) ARCH='s390x';; \
-    arm64) ARCH='arm64';; \
-    armhf) ARCH='armv7l';; \
-    i386) ARCH='x86';; \
-    *) echo "unsupported architecture"; exit 1 ;; \
-  esac && \
-    echo "Etc/UTC" > /etc/localtime && \
-	apt-get update && \
-	apt-get install -y --no-install-recommends ca-certificates wget python3 apt-utils && \
-	cd ~ && \
-	wget -q https://nodejs.org/download/release/v$NODE_VER/node-v$NODE_VER-linux-$ARCH.tar.gz && \
-	tar xf node-v$NODE_VER-linux-$ARCH.tar.gz && \
-	rm node-v$NODE_VER-linux-$ARCH.tar.gz && \
-	mv node-v$NODE_VER-linux-$ARCH /opt/node
+COPY --from=ruby /opt/ruby /opt/ruby   # HJH: space savings from --link dwarfed by media storage, not worth the hassle

-# Install Ruby 3.0
-ENV RUBY_VER="3.0.4"
-RUN apt-get update && \
-  apt-get install -y --no-install-recommends build-essential \
-    bison libyaml-dev libgdbm-dev libreadline-dev libjemalloc-dev \
-		libncurses5-dev libffi-dev zlib1g-dev libssl-dev && \
-	cd ~ && \
-	wget https://cache.ruby-lang.org/pub/ruby/${RUBY_VER%.*}/ruby-$RUBY_VER.tar.gz && \
-	tar xf ruby-$RUBY_VER.tar.gz && \
-	cd ruby-$RUBY_VER && \
-	./configure --prefix=/opt/ruby \
-	  --with-jemalloc \
-	  --with-shared \
-	  --disable-install-doc && \
-	make -j"$(nproc)" > /dev/null && \
-	make install && \
-	rm -rf ../ruby-$RUBY_VER.tar.gz ../ruby-$RUBY_VER
+ENV DEBIAN_FRONTEND="noninteractive" \
+    PATH="${PATH}:/opt/ruby/bin"

-ENV PATH="${PATH}:/opt/ruby/bin:/opt/node/bin"
-
-RUN npm install -g npm@latest && \
-	npm install -g yarn && \
-	gem install bundler && \
-	apt-get update && \
-	apt-get install -y --no-install-recommends git libicu-dev libidn11-dev \
-	libpq-dev shared-mime-info
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]

+WORKDIR /opt/mastodon
 COPY Gemfile* package.json yarn.lock /opt/mastodon/

-RUN cd /opt/mastodon && \
-  bundle config set --local deployment 'true' && \
-  bundle config set --local without 'development test' && \
-  bundle config set silence_root_warning true && \
-	bundle install -j"$(nproc)" && \
-	yarn install --pure-lockfile
+# hadolint ignore=DL3008
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends build-essential \
+        ca-certificates \
+        git \
+        libicu-dev \
+        libidn11-dev \
+        libpq-dev \
+        libjemalloc-dev \
+        zlib1g-dev \
+        libgdbm-dev \
+        libgmp-dev \
+        libssl-dev \
+        libyaml-0-2 \
+        ca-certificates \
+        libreadline8 \
+        python3 \
+        shared-mime-info && \
+    bundle config set --local deployment 'true' && \
+    bundle config set --local without 'development test' && \
+    bundle config set silence_root_warning true && \
+    bundle install -j"$(nproc)" && \
+    yarn install --pure-lockfile --network-timeout 600000

-FROM ubuntu:20.04
+FROM node:${NODE_VERSION}

-# Copy over all the langs needed for runtime
-COPY --from=build-dep /opt/node /opt/node
-COPY --from=build-dep /opt/ruby /opt/ruby
+ARG UID="991"
+ARG GID="991"

-# Add more PATHs to the PATH
-ENV PATH="${PATH}:/opt/ruby/bin:/opt/node/bin:/opt/mastodon/bin"
+COPY --from=ruby /opt/ruby /opt/ruby

-# Create the mastodon user
-ARG UID=991
-ARG GID=991
 SHELL ["/bin/bash", "-o", "pipefail", "-c"]
-RUN apt-get update && \
-	echo "Etc/UTC" > /etc/localtime && \
-	apt-get install -y --no-install-recommends whois wget && \
-	addgroup --gid $GID mastodon && \
-	useradd -m -u $UID -g $GID -d /opt/mastodon mastodon && \
-	echo "mastodon:$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 24 | mkpasswd -s -m sha-256)" | chpasswd && \
-	rm -rf /var/lib/apt/lists/*

-# Install mastodon runtime deps
-RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
-RUN apt-get update && \
-  apt-get -y --no-install-recommends install \
-	  libssl1.1 libpq5 imagemagick ffmpeg libjemalloc2 \
-	  libicu66 libidn11 libyaml-0-2 \
-	  file ca-certificates tzdata libreadline8 gcc tini apt-utils && \
-	ln -s /opt/mastodon /mastodon && \
-	gem install bundler && \
-	rm -rf /var/cache && \
-	rm -rf /var/lib/apt/lists/*
+ENV DEBIAN_FRONTEND="noninteractive" \
+    PATH="${PATH}:/opt/ruby/bin:/opt/mastodon/bin"
+
+# Ignoreing these here since we don't want to pin any versions and the Debian image removes apt-get content after use
+# hadolint ignore=DL3008,DL3009
+RUN apt-get update && \
+    echo "Etc/UTC" > /etc/localtime && \
+    groupadd -g "${GID}" mastodon && \
+    useradd -l -u "$UID" -g "${GID}" -m -d /opt/mastodon mastodon && \
+    apt-get -y --no-install-recommends install whois \
+        wget \
+        procps \
+        libssl1.1 \
+        libpq5 \
+        imagemagick \
+        ffmpeg \
+        libjemalloc2 \
+        libicu67 \
+        libidn11 \
+        libyaml-0-2 \
+        file \
+        ca-certificates \
+        tzdata \
+        libreadline8 \
+        tini && \
+    ln -s /opt/mastodon /mastodon
+
+# Note: no, cleaning here since Debian does this automatically
+# See the file /etc/apt/apt.conf.d/docker-clean within the Docker image's filesystem

-# Copy over mastodon source, and dependencies from building, and set permissions
 COPY --chown=mastodon:mastodon . /opt/mastodon
-COPY --from=build-dep --chown=mastodon:mastodon /opt/mastodon /opt/mastodon
+COPY --chown=mastodon:mastodon --from=build /opt/mastodon /opt/mastodon

-# Run mastodon services in prod mode
-ENV RAILS_ENV="production"
-ENV NODE_ENV="production"
-
-# Tell rails to serve static files
-ENV RAILS_SERVE_STATIC_FILES="true"
-ENV BIND="0.0.0.0"
+ENV RAILS_ENV="production" \
+    NODE_ENV="production" \
+    RAILS_SERVE_STATIC_FILES="true" \
+    BIND="0.0.0.0"

 # Set the run user
 USER mastodon
+WORKDIR /opt/mastodon

 # Precompile assets
-RUN cd ~ && \
-	OTP_SECRET=precompile_placeholder SECRET_KEY_BASE=precompile_placeholder rails assets:precompile && \
-	yarn cache clean
+RUN OTP_SECRET=precompile_placeholder SECRET_KEY_BASE=precompile_placeholder rails assets:precompile && \
+    yarn cache clean

 # Set the work dir and the container entry point
-WORKDIR /opt/mastodon
 ENTRYPOINT ["/usr/bin/tini", "--"]
 EXPOSE 3000 4000
--- a/docker-compose.yml
+++ b/docker-compose.yml
@ -3,7 +3,7 @@ services:
  db:
    restart: always
    image: postgres:14-alpine
-    shm_size: 256mb
+    shm_size: 512mb    # 256MB is much too small, yet this is probably small enough to avoid OOMs even with 2GB RAM
    networks:
      - internal_network
    healthcheck:
@ -23,6 +23,7 @@ services:
    volumes:
      - ./redis:/data

+  # disabled, as there are open issues related to it plus ES chews RAM and CPU
  # es:
  #   restart: always
  #   image: docker.elastic.co/elasticsearch/elasticsearch:7.17.4
@ -38,7 +39,6 @@ services:
  #     - "discovery.type=single-node"
  #     - "thread_pool.write.queue_size=1000"
  #   networks:
-  #      - external_network
  #      - internal_network
  #   healthcheck:
  #      test: ["CMD-SHELL", "curl --silent --fail localhost:9200/_cluster/health || exit 1"]
@ -56,7 +56,7 @@ services:

  web:
    build: .
-    image: tootsuite/mastodon:v3.5.5
+    image: tootsuite/mastodon:v4.0.2
    restart: always
    env_file: .env.production
    command: bash -c "rm -f /mastodon/tmp/pids/server.pid; bundle exec rails s -p 3000"
@ -77,7 +77,7 @@ services:

  streaming:
    build: .
-    image: tootsuite/mastodon:v3.5.5
+    image: tootsuite/mastodon:v4.0.2
    restart: always
    env_file: .env.production
    command: node ./streaming
@ -95,7 +95,7 @@ services:

  sidekiq:
    build: .
-    image: tootsuite/mastodon:v3.5.5
+    image: tootsuite/mastodon:v4.0.2
    restart: always
    env_file: .env.production
    command: bundle exec sidekiq