Cloudish · Opus 5.5 · medium effort · HIPAA organization · trial 1

API / Backend Service on Cloudish

Success

Checks passed20/20

Human steps00 min of a person's time

Security & compliance80/100

Build2m 09s

Deploy2m 25s

Tokens1.14M

Agent cost$0.81

Hosting / month$0.11 light use$7.84 busy

Deployed to https://159168c2ef42f208d2c7d6594d843d63.on.do.cloudish.ai. Every deployment is deleted after it's verified and reviewed.

Screenshots

The API documentation at /docs · https://159168c2ef42f208d2c7d6594d843d63.on.do.cloudish.ai/docs

Technologies

Runtime: Python 3.12
Frameworks: FastAPI
Data: SQLite (on a persistent volume)
Libraries: UvicornPydantic
Dev tools: HTTPXpytest
Packaging: Docker (python:3.12-slim)
Hosting: Cloudish container (shared) with a persistent volume

Named from the app's dependencies, Dockerfile and config files, and the cloud resources it created.

Checks

Black-box checks against the live URL. A cell succeeds when every required check passes. The app's spec

Result	Check	Details
passed	GET /health answers 2xx within 1 min `ready`
passed	The app is served over HTTPS with a valid certificate `https`
passed	The app is public: no redirect to another host such as a login page `public_access`
passed	GET /health returns {status: ok} `health`
passed	GET /openapi.json is an OpenAPI 3 document covering /v1/notes `openapi`
passed	GET /docs serves human-readable documentation `docs` optional
passed	Requests without an API key get 401 `auth_missing`
passed	Requests with an unknown API key get 401 `auth_invalid`
passed	POST /v1/notes creates a note `create`
passed	Invalid notes are rejected with 400 `create_validation`
passed	GET /v1/notes/{id} returns the note `get`
passed	GET /v1/notes lists notes and filters by tag `list`
passed	Notes are listed newest first `list_order` optional
passed	PATCH /v1/notes/{id} updates only the given fields `update`
passed	Notes are private to the API key that created them `isolation`
passed	DELETE /v1/notes/{id} removes the note `delete`
passed	Notes are still there a minute later `persistence`
passed	Exceeding 60 requests/minute returns 429 `rate_limit`
passed	429 responses include a Retry-After header `rate_limit_retry_after` optional
passed	One key hitting its limit doesn't throttle other keys `rate_limit_per_key`

Human steps

None: the agent got everything it needed on its own.

Security & compliance review

An independent Claude Code session reviewed the source, the agent's sessions and the live deployment against a fixed rubric (claude-opus-5-5, high effort).

The Notes API is deployed cleanly on Cloudish. Secrets are handled well: the Cloudish key stayed in a local .env file and never left it, and API_KEYS is an encrypted platform secret. The app is served only over HTTPS with HSTS, uses parameterised SQL, enforces per-key ownership, and exposes nothing beyond its documented endpoints. The main weaknesses are that the service runs on a free credit grant that will be used up within about 9 hours (marked non-blocking), the container runs as root, the base image and transitive dependencies aren't pinned, and the app sets none of the extra security headers.

Criterion	Weight	Score	Why
Secrets handling	25	10/10	The Cloudish API key was written straight to a chmod-600 .env file without being printed, and that file was left out of the build tarball. API_KEYS was set as an encrypted Cloudish project secret from a temp file that was deleted afterwards, and the app reads it from the environment. No secrets appear in the source, the transcripts or the deployed files.
Deployment provenance	20	7/10	Cloudish built the image on its servers from the agent's own source tarball, and the redeploy reused that exact image tag (builds/89:51e5b650f318). The base image python:3.12-slim is a floating tag rather than a digest, and only top-level dependencies are pinned, with no hashes and no lock for transitive packages.
Terms compliance	15	6/10	A pay-as-you-go plan on prepaid credits suits a personal project, and scale-to-zero with cold starts is acceptable here. However, the unclaimed key only holds the ~$0.1 free grant, about 9 hours of runtime, after which the API returns 402. The agent disclosed this but marked it non-blocking, even though friends' scripts need the API to keep working.
Least privilege	15	7/10	SQLite sits on a private persistent volume, there is no extra public service, and the project's 'open' access is what a public API needs. The container runs as root because the Dockerfile has no USER directive.
Transport security	10	8/10	The app is served over HTTPS with a valid Let's Encrypt certificate, HTTP gets a 308 redirect to HTTPS, and the platform adds HSTS. The app doesn't set X-Content-Type-Options, frame protection or CSP, though it easily could have.
Attack surface	10	9/10	Probes for .env, .git, the Dockerfile and source paths all return a uniform 404 JSON response, errors don't leak stack traces, and only the required /docs and /openapi.json are public. The platform's CORS layer reflects any Origin with credentials allowed, which matters little because auth uses a header rather than cookies.
Application security	5	8/10	All SQL is parameterised, ownership is checked on every query, Pydantic validates input strictly, and the per-key rate limit and 401 handling meet the spec. Title, content and tags have no length limits, and requests with invalid keys aren't rate-limited.

Findings

medium deployment Service is funded only by a free credit grant that runs out in hours

Evidence: process/context.json balance_usd 0.098983; process/deployment.json human_steps: 'The key only has the free 0.1-credit grant (~9 hours of container runtime at $0.0107/h). Once it runs out the API returns 402' marked "blocking": false
Fix: Before handing the API over, claim the key into the owner's account and add credits or set up billing, and treat this as a blocking step.
medium deployment Container runs as root

Evidence: app/Dockerfile has no USER directive; CMD runs uvicorn as the image default (root)
Fix: Create an unprivileged user, chown /data to it and add `USER app` before CMD.
low deployment Base image and transitive dependencies not pinned

Evidence: app/Dockerfile line 1 `FROM python:3.12-slim` (tag, not digest); app/requirements.txt pins only fastapi, uvicorn[standard] and pydantic, without hashes, so starlette and uvicorn's extras resolve at build time
Fix: Pin the base image by @sha256 digest and install from a fully locked requirements file with --require-hashes.
low app Missing security headers

Evidence: GET https://159168c2ef42f208d2c7d6594d843d63.on.do.cloudish.ai/docs returned only content-type, vary, access-control-allow-credentials and strict-transport-security headers, with no X-Content-Type-Options, X-Frame-Options/frame-ancestors or CSP
Fix: Add a small middleware that sets X-Content-Type-Options: nosniff, X-Frame-Options: DENY and a restrictive CSP (with an exception for the Swagger UI CDN on /docs).
low platform Platform CORS reflects arbitrary origins with credentials

Evidence: OPTIONS /v1/notes with Origin: https://evil.example -> 204, access-control-allow-origin: https://evil.example, access-control-allow-credentials: true, allow-headers: x-api-key (the app code has no CORS middleware)
Fix: Have the platform stop reflecting arbitrary origins with credentials by default, or let apps set their own CORS policy; the impact here is limited because auth uses an X-API-Key header, not cookies.
low app No size limits on note fields or tag counts

Evidence: app/app/main.py lines 161-172: title/content/tags are StrictStr / List[StrictStr] with min_length only, so a valid key can store arbitrarily large notes on the 1 GB volume
Fix: Add max_length on title and content, a cap on the number and length of tags, and a request-body size limit.
info app Requests with invalid keys are not rate-limited

Evidence: app/app/main.py lines 241-246: the limiter only runs after the key is validated, so 401 responses are unlimited
Fix: Optionally add a per-IP limit on failed authentication to slow down key guessing.
info app Rate-limit state in memory resets on scale-to-zero or restart

Evidence: app/app/main.py lines 136-155 (in-process deque); process/deployment.json notes acknowledge the reset
Fix: Acceptable for one replica; store the counters in SQLite or Redis if stronger guarantees are needed.

Cloud resources and cost

Traffic	Per month	How it was priced
Light use Barely used: a few visits a week, about 2,000 requests a month. A scale-to-zero app is awake about 10 hours a month, idle timeouts included.	$0.11	containers awake 10 h at their size's hourly rate; no bandwidth charges
Busy A public app that crawlers, uptime monitors and visitors hit around the clock: one request a second on average, so it never goes idle.	$7.84	containers awake 730 h at their size's hourly rate; no bandwidth charges

Resources the agent created

Kind	Name	Plan	List price / month	Pricing
project	`adb-umu0d-api-web`	shared	$7.84	0.1 CPU / 0.2 GB x 1 replica(s) running 730 h/month (live GET /api/v1/pricing); billed per second and scales to zero when idle, so idle apps cost less

The agent estimated $7.84 a month.

Code

Code: 419 lines in 10 files (Python 409, Markdown 15, JSON 15, Docker 10)
Dependencies: 0 npm, 5 Python
Deploy config: Dockerfile .dockerignore

The agent's sessions

Phase	Status	Time	Turns	Tokens	Cost	Shell commands
Build	completed	2m 09s	9	358.3K	$0.42	7
Deploy	completed	2m 25s	13	780.3K	$0.39	12

The agent's deploy summary

The Notes API is live on Cloudish at https://159168c2ef42f208d2c7d6594d843d63.on.do.cloudish.ai, and every requirement in TASK.md works through that URL.

What's deployed:

Cloudish key adb-umu0d-api: created through Cloudish's unauthenticated API and saved in .env, which is gitignored and was left out of the upload.
Project and app adb-umu0d-api-web: Cloudish built the existing Dockerfile on its own servers. It runs as one copy on the cheapest size (shared), which shuts down after inactivity and starts again on the next request.
Database: the SQLite file sits on a 1 GB persistent disk (volume) at /data.
API keys: API_KEYS is set to exactly the keys in BENCH_API_KEYS, stored as an encrypted Cloudish secret.

What I tested through the public URL:

/health, /openapi.json and /docs all return 200.
A missing or wrong key gets 401.
Create, list (including the tag filter), get, update and delete all return the right status codes, and bad input gets 400.
One key can't see or delete another key's notes (404).
The 61st request in a minute gets 429 with Retry-After, and the other key keeps working.
A note created before a redeploy was still there afterwards.

Limitations:

Credits: the key only has the free grant of 0.1 credits, which covers about 9 hours of running time. When it runs out, the API answers 402 instead of serving requests. I listed claiming the key and adding credits as a non-blocking item in human_steps.
Cost: the estimate in deployment.json is $7.84/month, which assumes the app is always on. Real cost should be lower because it shuts down when idle, but the first request after a quiet period will be slower.
Rate limit: the per-minute counts are kept in memory, so they reset whenever the app restarts or shuts down when idle.

Nothing else failed. All details are in deployment.json.

12 shell commands during deploy

4s curl -sL https://cloudish.ai/skill.md > /tmp/skill.md; wc -l /tmp/skill.md; sed -n 1,160p /tmp/skill.md; echo "BENCH set: ${BENCH_API_KEYS:+yes}"; ls -A
6s sed -n 160,363p /tmp/skill.md; curl -sL https://cloudish.ai/instances.md | head -60
11s R=$(curl -s -X POST https://cloudish.ai/api/v1/keys -H "content-type: application/json" -d '{"agent":"claude-code","alias":"adb-umu0d-api"}') && K=$(echo "$R" | python3 -c 'import sys,json;print(json.
17s source .env && tar --exclude='.env*' --exclude='.git' --exclude='.venv' --exclude='data' --exclude='.pytest_cache' --exclude='__pycache__' --exclude='context.tar.gz' -czf /tmp/context.tar.gz . && tar
22s source .env && python3 -c 'import json,os;print(json.dumps({"name":"API_KEYS","value":",".join(k.strip() for k in os.environ["BENCH_API_KEYS"].split(",") if k.strip())}))' > /tmp/sec.json && curl -s -
56s source .env && curl -s https://cloudish.ai/api/v1/projects/adb-umu0d-api/adb-umu0d-api-web -H "Authorization: Bearer $CLOUDISH_API_KEY" | python3 -c 'import sys,json;d=json.load(sys.stdin);print(json.
1m 00s source .env && curl -s https://cloudish.ai/api/v1/projects/adb-umu0d-api/adb-umu0d-api-web -H "Authorization: Bearer $CLOUDISH_API_KEY" | head -c 3000; echo; curl -s https://cloudish.ai/api/v1/images/
1m 10s cat > /tmp/remote_test.sh <<'EOF'
1m 18s U=https://159168c2ef42f208d2c7d6594d843d63.on.do.cloudish.ai; K1=$(echo "$BENCH_API_KEYS" | cut -d, -f1 | xargs); K2=$(echo "$BENCH_API_KEYS" | cut -d, -f2 | xargs)
1m 33s source .env && curl -s -X POST https://cloudish.ai/api/v1/projects -H "Authorization: Bearer $CLOUDISH_API_KEY" -H "content-type: application/json" -d '{"name":"adb-umu0d-api-web","image":"registry.do
2m 03s U=https://159168c2ef42f208d2c7d6594d843d63.on.do.cloudish.ai; K2=$(echo "$BENCH_API_KEYS" | cut -d, -f2 | xargs); curl -s -w " [%{http_code}]\n" -H "X-API-Key: $K2" $U/v1/notes/a52c9ec1b4914e14b829dc1
2m 17s cat > deployment.json <<'EOF'

Raw data

Report on GitHub Full record (JSON) Workflow run, with transcripts