fix(ci): deploy version tags and install instance controls
Development CI / check-and-package (push) Successful in 4m21s
GitHub Git mirror / mirror (push) Successful in 6s
Development CI / image (push) Successful in 22s
Development CI / deploy-development (push) Successful in 16s

This commit is contained in:
eezd committed 2026-11-02 15:37:54 +08:00
1 parent 6cbd604a28
commit 403824eccb
11 files changed
+335 -16

No files matched your search

+2 -2
View File
@@ -131,10 +131,10 @@ pnpm -C web build
Gitea 为公开主仓库,GitHub 为单向公开镜像;默认分支均为 main,日常开发使用 dev。
- dev push:容器内检查、Linux amd64 打包与隔离冒烟,通过后发布 `git.134333.xyz/eezd/dev/rawmark:YYYYMMDD-<8位SHA>`,按摘要部署 DEV。日期取提交时间 UTC,完整 SHA 保留在回执;同名异提交拒绝,已验证镜像重试复用。
- dev push:容器内检查、Linux amd64 打包与隔离冒烟,通过后发布 `git.134333.xyz/eezd/dev/rawmark:YYYYMMDD-<8位SHA>`,按已验收版本标签部署 DEV,摘要仅用于内部身份校验。日期取提交时间 UTC,完整 SHA 保留在回执;同名异提交拒绝,已验证镜像重试复用。
- main push:独立检查实际合并提交、Linux amd64 构建及容器冒烟,不发布镜像、不部署。仓库内受信 PR 只检查;外部 fork PR 不自动进入共享 Docker runner,不注入发布凭据。
- DEV 为 `/docker/gitea/dev/RawMark`,容器 `RawMark-dev`,Compose 项目 `rawmark-dev`,监听 `127.0.0.1:21001`,域名 `raw.134333.xyz`,raw-dev 域名停用。DEV 是唯一常驻实例,保存个人真实数据;CI 与冒烟使用隔离合成数据。数据独立;所有部署共用宿主实例 `.deploy.lock`,锁内核对 dev 最新 SHA,pending 未处理时拒绝覆盖。已有数据即使无旧容器也必须备份。
- Git 镜像独立于 CI 成败:push/create/delete、每小时对账和手动重试,共用 `/docker/gitea/mirrors/RawMark/.mirror.lock`,仅同步业务分支和 TAG 及完整历史,跟随源端删除。同步代码从 main 读取;`RAWMARK_MIRROR_ENABLED=true` 后启用,凭据统一使用 Gitea 账号级 Secret `ALL_GITHUB_TOKEN`(头像 → 设置 → Actions → 密钥),接入前配置并验证实际权限,仓库级同名项会覆盖账号级值。DEV 镜像不上传 GHCR。main 切换前保持该变量关闭。
- CI 凭据 `RAWMARK_GITEA_TOKEN` 需要仓库和 Packages 写权限。runner 保持容量一、`sweep: false`,只运行受信代码;工具链安装在任务容器。资源按任务标签清理,禁止全局 prune。
- 开发构建通过 `RAWMARK_BUILD_CHANNEL=development` 派生版本,前端与二进制一致;正式 `VERSION` 不因开发构建改变。开发包为 `RawMark_<版本>_linux_amd64.tar.gz`;历史正式产物不改名。安装包及单次传输保持既有 90,000,000 字节预算。
- 正式发布从受信 main 手动触发,使用日期哈希注解 TAG、六平台程序包与 Linux 双架构镜像;发布 ID 和候选 SHA 固定,发布记录与 .release.lock 位于持久镜像目录。GitHub 引用同步复用镜像锁,Pages Demo 不切换。双端发布和 GHCR 公开拉取验收后结束发布并标记预约完成,不部署 MAIN、不更新 DEV。DEV 批量管理入口为 `/docker/gitea/dev/manage.sh start|stop`,与 CD 共用实例锁,pending 阻止启动。
- 正式发布从受信 main 手动触发,使用日期哈希注解 TAG、六平台程序包与 Linux 双架构镜像;发布 ID 和候选 SHA 固定,发布记录与 .release.lock 位于持久镜像目录。GitHub 引用同步复用镜像锁,Pages Demo 不切换。双端发布和 GHCR 公开拉取验收后结束发布并标记预约完成,不部署 MAIN、不更新 DEV。DEV 启停入口为实例内 `start.sh` / `stop.sh`,由 `deploy/` 随 CD 安装和更新(755);与 CD 共用实例锁,pending 阻止启动。实例环境文件优先,拒绝覆盖本地改过的脚本。
+1 -1
View File
@@ -3,7 +3,7 @@ name: ${RAWMARK_PROJECT:?Set RAWMARK_PROJECT}
services:
rawmark:
container_name: ${RAWMARK_CONTAINER_NAME:?Set RAWMARK_CONTAINER_NAME}
image: ${RAWMARK_IMAGE:?Set a verified registry image digest}
image: ${RAWMARK_IMAGE:?Set a verified registry image tag}
ports:
- "${RAWMARK_BIND:-127.0.0.1}:${RAWMARK_PORT:?Set RAWMARK_PORT}:8080"
volumes:
+44
View File
@@ -0,0 +1,44 @@
#!/usr/bin/env bash
# Manage only this DEV instance under the deployment lock.
set -euo pipefail
if (( $# != 0 )); then
printf '用法:bash %s\n' "$0" >&2
exit 2
fi
instance_dir=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)
cd -- "$instance_dir"
for tool in docker flock env; do
command -v "$tool" >/dev/null 2>&1 || { printf '缺少命令:%s\n' "$tool" >&2; exit 1; }
done
exec 9>>.deploy.lock
flock -n 9 || { printf '实例正在部署或启停,请稍后重试\n' >&2; exit 1; }
files=()
for filename in compose.yaml compose.yml docker-compose.yaml docker-compose.yml; do
[[ ! -f $filename ]] || files+=("$filename")
done
if (( ${#files[@]} != 1 )); then
printf '实例必须有且只有一个 Compose 主文件,当前找到 %d 个\n' "${#files[@]}" >&2
exit 1
fi
# Keep Docker client access; resolve application settings only from instance files.
compose=(env -i "PATH=$PATH")
for variable in HOME DOCKER_CONFIG DOCKER_HOST DOCKER_CONTEXT DOCKER_TLS_VERIFY DOCKER_CERT_PATH XDG_RUNTIME_DIR SSH_AUTH_SOCK; do
if [[ -v $variable ]]; then
compose+=("$variable=${!variable}")
fi
done
compose+=(docker compose --project-directory "$instance_dir" -f "$instance_dir/${files[0]}")
# Explicit env files disable Compose's implicit .env, so include both in order.
for filename in .env images.env; do
if [[ -f $filename ]]; then
compose+=(--env-file "$instance_dir/$filename")
fi
done
if [[ -e deployment.pending.json ]]; then
printf '实例存在未完成部署,请先检查并恢复部署状态\n' >&2
exit 1
fi
"${compose[@]}" config --quiet
# Reuse the verified local image without building or pulling.
"${compose[@]}" up -d --no-build --pull never
printf '已启动实例:%s\n' "${instance_dir##*/}"
Executable
+40
View File
@@ -0,0 +1,40 @@
#!/usr/bin/env bash
# Manage only this DEV instance under the deployment lock.
set -euo pipefail
if (( $# != 0 )); then
printf '用法:bash %s\n' "$0" >&2
exit 2
fi
instance_dir=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)
cd -- "$instance_dir"
for tool in docker flock env; do
command -v "$tool" >/dev/null 2>&1 || { printf '缺少命令:%s\n' "$tool" >&2; exit 1; }
done
exec 9>>.deploy.lock
flock -n 9 || { printf '实例正在部署或启停,请稍后重试\n' >&2; exit 1; }
files=()
for filename in compose.yaml compose.yml docker-compose.yaml docker-compose.yml; do
[[ ! -f $filename ]] || files+=("$filename")
done
if (( ${#files[@]} != 1 )); then
printf '实例必须有且只有一个 Compose 主文件,当前找到 %d 个\n' "${#files[@]}" >&2
exit 1
fi
# Keep Docker client access; resolve application settings only from instance files.
compose=(env -i "PATH=$PATH")
for variable in HOME DOCKER_CONFIG DOCKER_HOST DOCKER_CONTEXT DOCKER_TLS_VERIFY DOCKER_CERT_PATH XDG_RUNTIME_DIR SSH_AUTH_SOCK; do
if [[ -v $variable ]]; then
compose+=("$variable=${!variable}")
fi
done
compose+=(docker compose --project-directory "$instance_dir" -f "$instance_dir/${files[0]}")
# Explicit env files disable Compose's implicit .env, so include both in order.
for filename in .env images.env; do
if [[ -f $filename ]]; then
compose+=(--env-file "$instance_dir/$filename")
fi
done
"${compose[@]}" config --quiet
# Preserve containers, networks, volumes and persistent data.
"${compose[@]}" stop
printf '已停止实例:%s\n' "${instance_dir##*/}"
+12 -3
View File
@@ -146,7 +146,7 @@ In addition to application backups, stop the service before upgrades and back up
1. Check the target version's behavior and data compatibility; retain the current image and deployment configuration.
2. Stop the old service and back up application data and any external vault.
3. Pull the selected prebuilt image digest and start with the same mounts. Source builds are optional contributor workflows, not managed production updates. Do not run old and new services concurrently.
3. Pull the selected prebuilt image version tag and start with the same mounts. Source builds are optional contributor workflows, not managed production updates. Do not run old and new services concurrently.
4. Verify container health, sign-in, notes, and attachments. Roll back using the matching previous image and data backup.
The Bearer authentication upgrade requires the frontend and backend to be released together. Old Cookie-based pages require signing in again. Startup creates the resource-ticket table and indexes through the existing SQLite initialization flow; notes, passwords, settings, and existing API tokens are retained. Third-party clients must follow the [current JSON protocol](api.md).
@@ -175,10 +175,19 @@ RAWMARK_CONTAINER_NAME=RawMark-dev
RAWMARK_BIND=127.0.0.1
RAWMARK_PORT=21001
RAWMARK_DATA_DIR=/docker/gitea/dev/RawMark/data
RAWMARK_IMAGE=git.134333.xyz/eezd/dev/rawmark@sha256:<verified-digest>
RAWMARK_IMAGE=git.134333.xyz/eezd/dev/rawmark:<verified-version>
```
Dev pushes deploy verified Gitea development digests after checks and isolated smoke tests. Main pushes and PRs only validate. Manual formal releases publish packages and Gitea/GHCR images with GitHub mirrors; they do not create MAIN or update DEV. `/docker/gitea/dev/manage.sh start|stop` manages first-level projects using instance `.deploy.lock` files and rejects starts with unresolved pending deployments.
Dev pushes deploy verified Gitea development version tags after checks and isolated smoke tests. Main pushes and PRs only validate. Manual formal releases publish packages and Gitea/GHCR images with GitHub mirrors; they do not create MAIN or update DEV. The instance-local controls share `.deploy.lock` with CD. Startup rejects unresolved pending deployments; stopping remains available after acquiring the lock.
```bash
bash /docker/gitea/dev/RawMark/start.sh
bash /docker/gitea/dev/RawMark/stop.sh
```
CD installs `deploy/start.sh` and `deploy/stop.sh` into the instance with mode 755 before stopping the service, on both first deployment and updates. The `.control-scripts.json` receipt tracks installed hashes; local edits or unrecognized existing scripts require review before deployment and are never silently overwritten. Initial manual setup can copy these two files after checking any existing entries.
Controls explicitly load `.env`, followed by `images.env` when present. Application and Compose variables from the caller cannot override those files; only PATH, HOME and the listed Docker client connection/configuration variables are retained. RawMark CD stores its verified image in `.env`; no separate `images.env` is needed. Startup reuses local images with `--no-build --pull never`; stopping preserves containers, networks, volumes and data. A successful start command means Compose accepted startup; use container health to confirm readiness.
Deployment stops the service and backs up complete data, including SQLite WAL, configuration and previous-image details before writing `deployment.pending.json`. Successful health checks write `deployment.json` and clear pending. A failed candidate is stopped while its backup and pending record remain; no automatic data restore or program downgrade occurs. Recovery and private-data migration require explicit authorization and compatibility review. Never run two instances sharing writable data.
+12 -3
View File
@@ -146,7 +146,7 @@ docker run -d --name rawmark --restart unless-stopped \
1. 确认目标版本的行为和数据兼容性,保留当前镜像及部署配置。
2. 停止旧服务,备份应用数据与外置笔记库。
3. 拉取选定的预构建镜像摘要,使用同一份挂载启动新实例;源码构建仅供贡献者选择,不用于托管生产更新。不要并行运行新旧服务。
3. 拉取选定的预构建镜像版本标签,使用同一份挂载启动新实例;源码构建仅供贡献者选择,不用于托管生产更新。不要并行运行新旧服务。
4. 核对容器健康、登录、笔记与附件。回退时使用配套的旧镜像与数据备份。
Bearer 认证升级需要前后端一起发布。旧 Cookie 页面不能继续使用,需重新登录;启动程序会沿现有 SQLite 初始化流程创建资源票据表和索引,笔记、密码、设置与已有 API 令牌保留。第三方客户端的 JSON 解包方式须遵循[当前协议](api.zh-CN.md)。
@@ -175,10 +175,19 @@ RAWMARK_CONTAINER_NAME=RawMark-dev
RAWMARK_BIND=127.0.0.1
RAWMARK_PORT=21001
RAWMARK_DATA_DIR=/docker/gitea/dev/RawMark/data
RAWMARK_IMAGE=git.134333.xyz/eezd/dev/rawmark@sha256:<已验证摘要>
RAWMARK_IMAGE=git.134333.xyz/eezd/dev/rawmark:<已验证版本>
```
DEV push 检查及隔离验收通过后自动部署 Gitea 开发镜像;main push 与 PR 只检查。正式 Release 手动发布程序包、Gitea/GHCR 镜像及 GitHub 镜像,不创建 MAIN 或更新 DEV。`/docker/gitea/dev/manage.sh start|stop` 批量管理一级项目,共用实例 `.deploy.lock`,pending 未处理时拒绝启动。
DEV push 检查及隔离验收通过后自动部署 Gitea 开发镜像;main push 与 PR 只检查。正式 Release 手动发布程序包、Gitea/GHCR 镜像及 GitHub 镜像,不创建 MAIN 或更新 DEV。实例启停脚本与 CD 共用 `.deploy.lock`。pending 未处理时拒绝启动,停止仍可在取得锁后执行。
```bash
bash /docker/gitea/dev/RawMark/start.sh
bash /docker/gitea/dev/RawMark/stop.sh
```
CD 在首次及后续部署停服前,将 `deploy/start.sh`、`deploy/stop.sh` 安装到实例并设置权限 755。`.control-scripts.json` 记录已安装脚本的哈希;遇到本地修改或无法识别的已有脚本会要求先审查,不会静默覆盖。首次手动配置也可在检查已有入口后复制这两个文件。
脚本显式依次加载 `.env` 和存在的 `images.env`,调用者的应用及 Compose 环境变量不能覆盖实例配置;只保留 PATH、HOME 及脚本列明的 Docker 客户端连接与配置变量。RawMark CD 将已验证镜像写入 `.env`,无需另建 `images.env`。启动通过 `--no-build --pull never` 复用本地镜像,停止保留容器、网络、卷和数据。启动命令成功表示 Compose 已接受启动,服务就绪仍需核对容器健康。
部署停止服务后备份完整数据(含 SQLite WAL)、配置和旧镜像信息,再写入 `deployment.pending.json`;健康检查通过才写入 `deployment.json` 并清除 pending。候选失败时停止候选、保留备份,不自动降级程序或恢复数据库。恢复和私人数据迁移需明确授权及兼容性核查;不得同时运行共享可写数据的新旧实例。
+2 -2
View File
@@ -112,10 +112,10 @@ Backend `internal/i18n/` preserves message IDs and parameters. Errors render at
Gitea is the public source repository; GitHub is a strict one-way Git mirror. Both use main as the default branch and dev for daily development.
- A dev push checks, packages and smoke-tests Linux amd64 in CI containers, publishes only `git.134333.xyz/eezd/dev/rawmark:YYYYMMDD-<8-char-SHA>`, then deploys its verified digest. The date comes from the commit timestamp in UTC. Receipts retain the full SHA; conflicting versions fail and verified images are reused on retry.
- A dev push checks, packages and smoke-tests Linux amd64 in CI containers, publishes only `git.134333.xyz/eezd/dev/rawmark:YYYYMMDD-<8-char-SHA>`, then deploys its verified version tag. The date comes from the commit timestamp in UTC. Receipts retain the full SHA; conflicting versions fail and verified images are reused on retry.
- A main push independently checks and smoke-tests the actual merged commit without publishing images or deploying. Trusted same-repository PRs only check; external fork PRs do not automatically enter the shared Docker runner or receive publication credentials.
- DEV lives at `/docker/gitea/dev/RawMark`, container `RawMark-dev`, explicit Compose project `rawmark-dev`, loopback port 21001 and domain `raw.134333.xyz`; raw-dev is retired. DEV is the only persistent instance and holds private user data; CI smoke tests use isolated synthetic data. All deployment entrypoints share the instance `.deploy.lock`; stale dev commits are skipped and unresolved pending deployments block updates. Existing data is backed up even when no old container exists.
- Git mirroring runs independently on push/create/delete, hourly reconciliation and manual retry, sharing `/docker/gitea/mirrors/RawMark/.mirror.lock`. Only business branches, tags and complete history are mirrored, including deletions. Control code comes from main. Enable `RAWMARK_MIRROR_ENABLED=true` after the main cutover and configure the account-level `ALL_GITHUB_TOKEN` in Gitea user Settings → Actions → Secrets with repository/workflow write access. DEV images never go to GHCR.
- `RAWMARK_GITEA_TOKEN` requires repository and Packages write access. Keep runner capacity one and sweep disabled; only trusted code runs on this Docker-socket runner. Install toolchains inside CI containers and clean only resources labeled for the current attempt.
- `RAWMARK_BUILD_CHANNEL=development` derives a consistent frontend/binary version without editing the formal VERSION file. Development archives use `RawMark_<version>_linux_amd64.tar.gz`; historical release artifacts retain their names. Keep the existing 90,000,000-byte transfer budget.
- Formal releases run manually from trusted main, require successful main-push CI, and publish six native packages plus Linux amd64/arm64 images with date/hash versions. Fixed release IDs retain resumable checkpoints under the persistent release lock. After both releases and public GHCR digest pulls pass, the reservation is marked complete without deploying or restarting any instance. Use `/docker/gitea/dev/manage.sh start|stop` for bulk DEV management with instance locks and pending-start protection. Keep Pages unchanged.
- Formal releases run manually from trusted main, require successful main-push CI, and publish six native packages plus Linux amd64/arm64 images with date/hash versions. Fixed release IDs retain resumable checkpoints under the persistent release lock. After both releases and public GHCR digest pulls pass, the reservation is marked complete without deploying or restarting any instance. Use `bash /docker/gitea/dev/RawMark/start.sh` or `bash /docker/gitea/dev/RawMark/stop.sh` to control only this instance, with the deployment lock and pending-start protection. CD installs and updates these executable scripts from `deploy/`, preserving locally modified entries for review. Keep Pages unchanged.
+27 -2
View File
@@ -4,6 +4,7 @@
import argparse
import contextlib
import fcntl
import hashlib
import json
import os
from pathlib import Path
@@ -46,12 +47,35 @@ def save(path, data):
temporary.unlink(missing_ok=True)
def install_controls(directory):
# Recognize the original templates; subsequent updates verify the installed receipt.
legacy = {'start.sh': 'b4709f36a66b428671a09f373f640a99259845924779dcb122f5cd9c17e85ab6',
'stop.sh': '83348a7781b7d738a715bde51387951041a5cfcd31796805422ef83139a21571'}
receipt = directory / '.control-scripts.json'
previous = json.loads(receipt.read_text()) if receipt.exists() else {}
sources = Path(__file__).resolve().parents[1] / 'deploy'
scripts = {name: (sources / name).read_bytes() for name in legacy}
hashes = {name: hashlib.sha256(data).hexdigest() for name, data in scripts.items()}
# Check both entries before replacing either; preserve custom or locally edited scripts.
for name in scripts:
target = directory / name
require(not target.is_symlink(), f'Refusing to replace symlink: {target}')
if target.exists():
current = hashlib.sha256(target.read_bytes()).hexdigest()
require(current in (legacy[name], hashes[name], previous.get(name)),
f'Locally modified control script requires review: {target}')
for name, data in scripts.items():
save(directory / name, data)
(directory / name).chmod(0o755)
save(receipt, (json.dumps(hashes, indent=2) + '\n').encode())
def deployment(directory, channel, image, commit, version, client_container=None):
directory = Path(directory).resolve(strict=True)
require(channel == 'development', 'Only the persistent DEV instance supports deployment')
cfg = settings(directory, channel)
registry = GITEA_DEV_IMAGE
require(re.fullmatch(re.escape(registry) + r'@sha256:[0-9a-f]{64}', image), 'Expected an immutable registry digest')
require(image == registry + ':' + version, 'Expected the selected development version tag')
require(re.fullmatch(r'[0-9a-f]{40}', commit), 'Expected a full commit SHA')
# Serializes actual deployments, including manually started jobs and CLI use.
with (directory / '.deploy.lock').open('a') as lock:
@@ -85,6 +109,7 @@ def deployment(directory, channel, image, commit, version, client_container=None
compose = ['docker', 'compose', '--project-name', cfg['RAWMARK_PROJECT'], '--env-file', str(directory / '.env'),
'-f', str(directory / 'compose.yml')]
run(*compose, 'config', '--quiet', env=env)
install_controls(directory)
backup = None
if previous:
run('docker', 'stop', '--time', '15', previous['Id'])
@@ -148,7 +173,7 @@ def main():
parser.add_argument('--version', required=True)
parser.add_argument('--client-container')
args = parser.parse_args()
require(args.image and args.commit, 'Development requires an image digest and commit')
require(args.image and args.commit, 'Development requires an image version tag and commit')
deployment(args.directory, args.channel, args.image, args.commit, args.version, args.client_container)
+2 -2
View File
@@ -532,7 +532,7 @@ def promote_development(local_image):
input=token, text=True, check=True)
existing = registry_digest(GITEA_DEV_IMAGE, build['version'])
if existing:
reference = GITEA_DEV_IMAGE + '@' + existing
reference = GITEA_DEV_IMAGE + ':' + build['version']
run('docker', 'pull', reference)
info = json.loads(run('docker', 'image', 'inspect', reference))[0]
labels = info['Config'].get('Labels') or {}
@@ -561,7 +561,7 @@ def promote_development(local_image):
run('docker', 'buildx', 'imagetools', 'create', '--prefer-index=false', '--tag',
GITEA_DEV_IMAGE + ':' + tag, GITEA_DEV_IMAGE + '@' + value)
require(registry_digest(GITEA_DEV_IMAGE, tag) == value, 'Development tag digest changed')
output('image', GITEA_DEV_IMAGE + '@' + value)
output('image', GITEA_DEV_IMAGE + ':' + build['version'])
output('version', build['version'])
finally:
run('docker', 'image', 'rm', candidate)
+29 -1
View File
@@ -204,7 +204,7 @@ class DeploymentTests(unittest.TestCase):
(self.directory / '.env').write_text(
f'RAWMARK_CHANNEL=development\nRAWMARK_PROJECT=rawmark-ci-deploy\n'
f'RAWMARK_BIND=127.0.0.1\nRAWMARK_PORT=21899\nRAWMARK_DATA_DIR={self.directory}/data\n')
self.image = release.GITEA_DEV_IMAGE + '@' + DIGEST
self.image = release.GITEA_DEV_IMAGE + ':0.1.1'
self.previous = {'Id': 'old-id', 'Image': 'old-image', 'State': {'Running': True},
'Mounts': [{'Destination': '/data', 'Type': 'bind', 'Source': str(self.directory / 'data')}]}
self.image_info = {'Id': 'new-image', 'Config': {'User': '65532:65532', 'Labels': {
@@ -236,6 +236,24 @@ class DeploymentTests(unittest.TestCase):
'test': {'NetworkID': 'network', 'IPAddress': '127.0.0.1'}}}}])
return ''
def test_existing_verified_build_outputs_a_tag(self):
build={'commit':SHA,'version':'0.1.1'}
info=json.dumps([{'Config':{'Labels':{'org.opencontainers.image.revision':SHA,
'org.opencontainers.image.version':'0.1.1'}}}])
with patch.object(release,'verify_bundle',return_value=build), \
patch.object(release.subprocess,'run'), patch.object(release,'registry_digest',return_value=DIGEST), \
patch.object(release,'run',return_value=info) as run, patch.object(release,'output') as output:
release.promote_development('synthetic')
output.assert_any_call('image',release.GITEA_DEV_IMAGE+':0.1.1')
self.assertEqual(run.call_args_list[0].args,('docker','pull',release.GITEA_DEV_IMAGE+':0.1.1'))
def test_digest_and_other_version_tags_cannot_deploy(self):
for image in (release.GITEA_DEV_IMAGE + '@' + DIGEST, release.GITEA_DEV_IMAGE + ':other'):
with self.subTest(image=image), patch.object(deploy, 'run') as docker:
with self.assertRaisesRegex(ValueError, 'version tag'):
deploy.deployment(self.directory, 'development', image, SHA, '0.1.1')
docker.assert_not_called()
def test_channel_mismatch_cannot_touch_docker(self):
with patch.object(deploy, 'run') as docker:
with self.assertRaisesRegex(ValueError, 'channel'):
@@ -277,6 +295,16 @@ class DeploymentTests(unittest.TestCase):
start = next(i for i,c in enumerate(self.commands) if c[:2] == ('docker', 'compose') and 'up' in c)
self.assertLess(stop, start)
self.assertIn('RAWMARK_IMAGE=' + self.image, (self.directory / '.env').read_text())
self.assertTrue((self.directory / 'start.sh').stat().st_mode & 0o111)
self.assertTrue((self.directory / 'stop.sh').stat().st_mode & 0o111)
def test_custom_control_script_is_preserved_before_stopping(self):
(self.directory / 'stop.sh').write_text('custom behavior\n')
with patch.object(deploy, 'run', side_effect=self.command):
with self.assertRaisesRegex(ValueError, 'Locally modified'):
deploy.deployment(self.directory, 'development', self.image, SHA, '0.1.1')
self.assertFalse(any(c[:2] == ('docker', 'stop') for c in self.commands))
self.assertEqual((self.directory / 'stop.sh').read_text(), 'custom behavior\n')
def test_existing_data_is_backed_up_without_a_container(self):
def without_container(*args, **kwargs):
+164
View File
@@ -0,0 +1,164 @@
"""Verify instance controls with synthetic files and no container mutations."""
import fcntl
import hashlib
import json
import os
from pathlib import Path
import shutil
import subprocess
import sys
import tempfile
import unittest
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
import deploy
SOURCES = Path(__file__).resolve().parents[2] / 'deploy'
class InstanceControlTests(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory(prefix='rawmark controls ')
self.addCleanup(self.tmp.cleanup)
self.directory = Path(self.tmp.name)
self.bin = self.directory / 'bin'
self.bin.mkdir()
for name in ('start.sh', 'stop.sh'):
shutil.copy2(SOURCES / name, self.directory / name)
(self.directory / 'compose.yml').write_text(
'name: ${RAWMARK_PROJECT:?missing project}\nservices:\n app:\n'
' image: ${RAWMARK_IMAGE:?missing image}\n environment:\n'
' APP_SETTING: ${APP_SETTING:?missing setting}\n'
' volumes:\n - ${RAWMARK_DATA_DIR:?missing data}:/data\n')
(self.directory / '.env').write_text(
'RAWMARK_PROJECT=rawmark-controls\nRAWMARK_IMAGE=example.invalid/verified:old\n'
f'APP_SETTING=from-dotenv\nRAWMARK_DATA_DIR={self.directory}/data\n')
(self.directory / 'data').mkdir()
(self.directory / 'data/keep').write_text('persistent')
(self.bin / 'docker').write_text(f'#!{sys.executable}\n' + '''
import json
import os
from pathlib import Path
import subprocess
import sys
directory = Path(__file__).resolve().parents[1]
args = sys.argv[1:]
with (directory / 'calls.jsonl').open('a') as log:
log.write(json.dumps({'args': args, 'env': dict(os.environ)}) + '\\n')
if 'config' in args and (directory / 'real-docker').exists():
real = (directory / 'real-docker').read_text()
result = subprocess.run([real, *args[:-1], '--format', 'json'], capture_output=True, text=True)
(directory / 'resolved.json').write_text(result.stdout)
sys.stderr.write(result.stderr)
sys.exit(result.returncode)
status = directory / 'failure-code'
if 'config' not in args and (directory / 'action-failure-code').exists():
status = directory / 'action-failure-code'
sys.exit(int(status.read_text()) if status.exists() else 0)
''')
(self.bin / 'docker').chmod(0o755)
def invoke(self, name, **environment):
return subprocess.run(['bash', str(self.directory / name)], cwd='/',
env={**os.environ, 'PATH': str(self.bin) + ':' + os.environ['PATH'], **environment},
capture_output=True, text=True)
def calls(self):
path = self.directory / 'calls.jsonl'
return [json.loads(line) for line in path.read_text().splitlines()] if path.exists() else []
def test_start_and_stop_use_instance_files_and_preserve_data(self):
(self.directory / 'images.env').write_text('RAWMARK_IMAGE=example.invalid/verified:new\n')
for name, action in (('start.sh', ['up', '-d', '--no-build', '--pull', 'never']),
('stop.sh', ['stop'])):
with self.subTest(name=name):
result = self.invoke(name, RAWMARK_PROJECT='other-project', RAWMARK_IMAGE='unverified',
RAWMARK_DATA_DIR='/other-data', APP_SETTING='caller',
COMPOSE_FILE='/wrong.yml', COMPOSE_PROJECT_NAME='other',
COMPOSE_PROFILES='extra', COMPOSE_DISABLE_ENV_FILE='true')
self.assertEqual(result.returncode, 0, result.stderr)
call = self.calls()[-1]
self.assertEqual(call['args'][-len(action):], action)
files = [call['args'][i + 1] for i, arg in enumerate(call['args']) if arg == '--env-file']
self.assertEqual(files, [str(self.directory / '.env'), str(self.directory / 'images.env')])
self.assertEqual(call['args'][call['args'].index('--project-directory') + 1], str(self.directory))
self.assertFalse(any(key.startswith(('RAWMARK_', 'COMPOSE_')) or key == 'APP_SETTING'
for key in call['env']))
self.assertEqual((self.directory / 'data/keep').read_text(), 'persistent')
def test_docker_client_configuration_is_preserved(self):
result = self.invoke('stop.sh', DOCKER_CONFIG='/private/docker-config')
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(self.calls()[-1]['env']['DOCKER_CONFIG'], '/private/docker-config')
def test_pending_blocks_start_but_allows_stop(self):
(self.directory / 'deployment.pending.json').write_text('{}')
self.assertNotEqual(self.invoke('start.sh').returncode, 0)
self.assertEqual(self.calls(), [])
self.assertEqual(self.invoke('stop.sh').returncode, 0)
self.assertEqual(self.calls()[-1]['args'][-1], 'stop')
def test_lock_blocks_both_controls(self):
with (self.directory / '.deploy.lock').open('a') as lock:
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
for name in ('start.sh', 'stop.sh'):
self.assertNotEqual(self.invoke(name).returncode, 0)
self.assertEqual(self.calls(), [])
def test_invalid_compose_count_rejects_before_docker(self):
(self.directory / 'compose.yaml').write_text('services: {}')
for name in ('start.sh', 'stop.sh'):
self.assertNotEqual(self.invoke(name).returncode, 0)
self.assertEqual(self.calls(), [])
def test_config_failure_prevents_action_and_propagates_status(self):
(self.directory / 'failure-code').write_text('17')
for name in ('start.sh', 'stop.sh'):
result = self.invoke(name)
self.assertEqual(result.returncode, 17)
self.assertEqual(self.calls()[-1]['args'][-2:], ['config', '--quiet'])
def test_action_failure_propagates_without_success_message(self):
(self.directory / 'action-failure-code').write_text('19')
for name in ('start.sh', 'stop.sh'):
result = self.invoke(name)
self.assertEqual(result.returncode, 19)
self.assertEqual(result.stdout, '')
def test_real_compose_loads_dotenv_and_image_override_without_caller_overrides(self):
docker = shutil.which('docker')
if not docker or subprocess.run([docker, 'compose', 'version'], capture_output=True).returncode:
self.skipTest('Docker Compose is not installed')
(self.directory / 'real-docker').write_text(docker)
(self.directory / 'images.env').write_text('RAWMARK_IMAGE=example.invalid/verified:new\n')
for name in ('start.sh', 'stop.sh'):
result = self.invoke(name, RAWMARK_PROJECT='wrong', RAWMARK_IMAGE='wrong',
RAWMARK_DATA_DIR='/wrong', APP_SETTING='wrong')
self.assertEqual(result.returncode, 0, result.stderr)
resolved = json.loads((self.directory / 'resolved.json').read_text())
self.assertEqual(resolved['name'], 'rawmark-controls')
self.assertEqual(resolved['services']['app']['image'], 'example.invalid/verified:new')
self.assertEqual(resolved['services']['app']['environment']['APP_SETTING'], 'from-dotenv')
self.assertEqual(resolved['services']['app']['volumes'][0]['source'], str(self.directory / 'data'))
class ControlInstallationTests(unittest.TestCase):
def test_fresh_install_and_update_preserve_local_edits(self):
with tempfile.TemporaryDirectory() as temporary:
directory = Path(temporary)
deploy.install_controls(directory)
for name in ('start.sh', 'stop.sh'):
self.assertEqual((directory / name).stat().st_mode & 0o777, 0o755)
# A previously managed version can be updated using its recorded digest.
(directory / 'start.sh').write_text('previous managed version')
receipt = json.loads((directory / '.control-scripts.json').read_text())
receipt['start.sh'] = hashlib.sha256((directory / 'start.sh').read_bytes()).hexdigest()
(directory / '.control-scripts.json').write_text(json.dumps(receipt))
deploy.install_controls(directory)
self.assertEqual((directory / 'start.sh').read_bytes(), (SOURCES / 'start.sh').read_bytes())
(directory / 'stop.sh').write_text('custom user behavior')
with self.assertRaisesRegex(ValueError, 'Locally modified'):
deploy.install_controls(directory)
self.assertEqual((directory / 'stop.sh').read_text(), 'custom user behavior')