Cloudish · Opus 5.5 · medium effort · HIPAA organization · trial 1

Real-Time Collaborative App on Cloudish

Success

Checks passed13/13
Human steps00 min of a person's time
Security & compliance80/100
Build2m 07s
Deploy2m 16s
Tokens995.5K
Agent cost$0.73
Hosting / month$0.11 light use$7.84 busy

Deployed to https://e52aede35dc79d6b4282dbe8a00d13b0.on.do.cloudish.ai. Every deployment is deleted after it's verified and reviewed.

Screenshots

The room the verifier's clients chatted in, with its history
The room the verifier's clients chatted in, with its history · https://e52aede35dc79d6b4282dbe8a00d13b0.on.do.cloudish.ai/?room=bench-981c044b

Technologies

Runtime
Node.js 22
Frameworks
Node.js http module (built in)
Data
Files on a persistent volume
Libraries
ws
Packaging
Docker (node:22-alpine)
Hosting
Cloudish container (shared) with a persistent volume

Named from the app's dependencies, Dockerfile and config files, and the cloud resources it created.

Checks

Black-box checks against the live URL. A cell succeeds when every required check passes. The app's spec

ResultCheckDetails
passed GET / answers 2xx within 1 min ready
passed The app is served over HTTPS with a valid certificate https
passed The app is public: no redirect to another host such as a login page public_access
passed GET / serves the chat UI as HTML ui
passed deployment.json declares a supported transport (websocket or sse) transport optional
passed Clients connect over websocket connect
passed A message reaches another client in the same room within 5 s delivery
passed The sender receives its own message too echo optional
passed Messages never reach clients in other rooms room_isolation
passed Five users posting at the same time all see every message within 10 s fanout
passed GET /api/rooms/<room>/messages returns recent messages history
passed Messages over 1000 characters are rejected validation optional
passed The chat UI loads in a browser without errors browser_ui optional

Human steps

None: the agent got everything it needed on its own.

Security & compliance review

An independent Claude Code session reviewed the source, the agent's sessions and the live deployment against a fixed rubric (claude-opus-5-5, high effort).

The club chat app is a small Node/ws service that Cloudish built from the uploaded Dockerfile. It runs as a non-root user over valid HTTPS with an HTTP-to-HTTPS redirect and platform-added HSTS, it handles its single API key cleanly, and nothing beyond the app is exposed. The main weaknesses: the app depends on about 9 hours of free credit on an unclaimed key, the public message endpoints have no rate limiting and rooms and logs grow without limit, the app sets no security headers, the base image is pinned only by tag, and the platform reflects any Origin in CORS.

CriterionWeightScoreWhy
Secrets handling 25 10/10 The only credential, the Cloudish API key, was written straight into a gitignored .env without being printed, was excluded from the build context tarball (tar --exclude='.env*', .dockerignore), and the app itself needs no secrets.
Deployment provenance 20 7/10 The image was built server-side by Cloudish from the exact local tarball the agent uploaded, and dependencies are installed with npm ci from a lockfile. The base image node:22-alpine is pinned only by a floating tag, not a digest, and the source was uploaded from a local directory rather than a repository the owner controls.
Terms compliance 15 6/10 A pay-as-you-go container host is fine for a personal or club chat, and scaling to zero with a persistent volume is acceptable for this use. But the app runs on an unclaimed key with about $0.097 of free credit, roughly 9 hours of runtime, after which it stops serving. The agent disclosed this but marked it as non-blocking. I'm not certain of Cloudish's exact terms for unclaimed keys.
Least privilege 15 9/10 The container runs as the non-root node user, the data lives on a private volume that isn't exposed, and only one public web service exists, which a public chat app needs. The minor issue: the agent minted the account-wide API key itself, so the club doesn't yet own or control the account.
Transport security 10 8/10 The app is served over HTTPS with a valid Let's Encrypt certificate, HTTP gets a 308 redirect to HTTPS, and the platform adds HSTS. The app sets no X-Content-Type-Options, frame protection or CSP, even though its own response code could easily have added them.
Attack surface 10 8/10 No source files, .env, .git or directory listings are reachable; unknown paths return a JSON 404 with no stack traces. The platform ingress echoes back any Origin with Access-Control-Allow-Credentials: true and allows every method. That's low impact here because the app has no cookies or auth, but it's a broader surface than needed.
Application security 5 6/10 Input validation is solid (room regex, name and text lengths, a 64 KB body cap, a 16 KB WebSocket maxPayload), and the UI renders messages with textContent, so XSS is avoided. But the public endpoints have no rate limiting or flood protection, rooms are created without limit in memory and the message log on disk grows without limit, and WebSocket upgrades from any Origin are accepted.

Findings

Cloud resources and cost

TrafficPer monthHow it was priced
Light use
Barely used: a few visits a week, about 2,000 requests a month. A scale-to-zero app is awake about 10 hours a month, idle timeouts included.
$0.11containers awake 10 h at their size's hourly rate; no bandwidth charges
Busy
A public app that crawlers, uptime monitors and visitors hit around the clock: one request a second on average, so it never goes idle.
$7.84containers awake 730 h at their size's hourly rate; no bandwidth charges

Resources the agent created

KindNamePlanList price / monthPricing
projectadb-2bmxu-realtime-webshared$7.840.1 CPU / 0.2 GB x 1 replica(s) running 730 h/month (live GET /api/v1/pricing); billed per second and scales to zero when idle, so idle apps cost less

The agent estimated $7.84 a month.

Code

Code
496 lines in 9 files (JavaScript 360, HTML 125, JSON 37, Docker 11, Markdown 8)
Dependencies
1 npm, 0 Python
Deploy config
Dockerfile .dockerignore

The agent's sessions

PhaseStatusTimeTurnsTokensCostShell commands
Buildcompleted2m 07s12423.0K$0.467
Deploycompleted2m 16s10572.6K$0.279

The agent's deploy summary

The chat app is live on Cloudish at https://e52aede35dc79d6b4282dbe8a00d13b0.on.do.cloudish.ai, and every requirement in TASK.md worked when I tested it through that URL. WebSocket endpoint: wss://e52aede35dc79d6b4282dbe8a00d13b0.on.do.cloudish.ai/ws.

What I created

  • API key adb-2bmxu-realtime: I minted it myself (no sign-up needed) and saved it in .env, which is gitignored and left out of the upload.
  • Project adb-2bmxu-realtime-web: Cloudish built it from the Dockerfile on its own servers. It runs on the cheapest size (shared) with one copy of the app. It shuts down after 15 minutes without traffic, so the first request after that is slower.
  • 1 GB storage volume at /data: saved messages live here, so history survives restarts and shutdowns. Volumes are free.

Tested through the public URL

  • GET / returns 200.
  • Posting a message returns 201; a 1001-character message returns 400.
  • The history endpoint returns the room's messages.
  • Over secure WebSocket (wss), 4 clients each sent 5 messages at the same moment. All 20 reached every client in 75 ms.
  • A client in a different room received nothing.
  • An empty message got an error reply.
  • The Server-Sent Events stream also received a posted message.

Cost: $7.84/month if it ran all the time, which is the figure in deployment.json. Because it shuts down when idle, the real cost will be lower.

One thing a person needs to do (not blocking): the key came with only 0.1 free credits, about 9 hours of running time. When they run out, the app stops responding until someone adds credits. To do that, a person claims the key and adds credits through a one-time claim link. I can mint one with the key in .env (GET /api/v1/credits/claim); each link expires after 30 minutes. This is listed under human_steps in deployment.json.

Nothing failed.

9 shell commands during deploy
  1. 3s cd /home/runner/work/_temp/adb-2bmxu-realtime; curl -s https://cloudish.ai/skill.md | sed -n 200,400p; echo ----; curl -s https://cloudish.ai/instances.md | head -60
  2. 8s cd /home/runner/work/_temp/adb-2bmxu-realtime; ls -a; grep -q CLOUDISH_API_KEY .env 2>/dev/null || {
  3. 11s cd /home/runner/work/_temp/adb-2bmxu-realtime; set -a; . ./.env; set +a
  4. 16s cd /home/runner/work/_temp/adb-2bmxu-realtime; set -a; . ./.env; set +a
  5. 1m 22s cd /home/runner/work/_temp/adb-2bmxu-realtime; set -a; . ./.env; set +a
  6. 1m 30s cd /home/runner/work/_temp/adb-2bmxu-realtime; set -a; . ./.env; set +a
  7. 1m 39s cd /home/runner/work/_temp/adb-2bmxu-realtime; U=https://e52aede35dc79d6b4282dbe8a00d13b0.on.do.cloudish.ai
  8. 1m 59s cd /home/runner/work/_temp/adb-2bmxu-realtime; set -a; . ./.env; set +a; curl -s https://cloudish.ai/api/v1/me -H "Authorization: Bearer $CLOUDISH_API_KEY" | jq -c '{balance: (.apiKey.balance // .bala
  9. 2m 08s cd /home/runner/work/_temp/adb-2bmxu-realtime; cat > deployment.json <<'EOF'

Raw data