<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>debugging · wleeaf field notes</title><link>https://articles.wleeaf.dev/tags/debugging/</link><description>Hands-on guides and build logs from the wleeaf studio: self-hosting, Linux, networking, Docker, the command line and reverse engineering.</description><language>en-us</language><lastBuildDate>Sun, 29 Mar 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://articles.wleeaf.dev/tags/debugging/feed.xml" rel="self" type="application/rss+xml"/><item><title>Debugging Running Docker Containers Without Restarting Them</title><link>https://articles.wleeaf.dev/posts/docker-debugging-running-containers/</link><pubDate>Sun, 29 Mar 2026 00:00:00 +0000</pubDate><guid>https://articles.wleeaf.dev/posts/docker-debugging-running-containers/</guid><description>exec, logs, inspect, and nsenter. Four tools for figuring out what is happening inside a container without touching it.</description><category>docker</category><category>debugging</category><category>linux</category><content:encoded><![CDATA[<p>Something is wrong with your container. It is running but not behaving. You could rebuild and redeploy, but that might hide the problem. Here is how to investigate a running container without restarting it, using <code>docker exec</code>, <code>docker logs</code>, <code>docker inspect</code>, <code>docker cp</code>, <code>nsenter</code> and <code>docker diff</code>.</p>
<h2 id="docker-exec">docker exec<a class="anchor" href="#docker-exec" aria-label="Link to “docker exec”">#</a></h2>
<p>The obvious one. Drop into a shell:</p>
<div class="code">
<div class="code-head"><span class="code-lang">bash</span><button type="button" class="code-copy" data-copy-code hidden><svg viewBox="0 0 24 24" aria-hidden="true"><rect x="8" y="8" width="12" height="12" rx="2.5"/><path d="M16 8V6.5A2.5 2.5 0 0 0 13.5 4h-7A2.5 2.5 0 0 0 4 6.5v7A2.5 2.5 0 0 0 6.5 16H8"/></svg><span>Copy</span></button></div>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">docker <span class="nb">exec</span> -it mycontainer sh</span></span></code></pre></div>
</div>
<p>Use <code>sh</code> not <code>bash</code> because Alpine images do not have bash. Once inside, you can poke around:</p>
<div class="code">
<div class="code-head"><span class="code-lang">bash</span><button type="button" class="code-copy" data-copy-code hidden><svg viewBox="0 0 24 24" aria-hidden="true"><rect x="8" y="8" width="12" height="12" rx="2.5"/><path d="M16 8V6.5A2.5 2.5 0 0 0 13.5 4h-7A2.5 2.5 0 0 0 4 6.5v7A2.5 2.5 0 0 0 6.5 16H8"/></svg><span>Copy</span></button></div>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># Check what processes are running</span>
</span></span><span class="line"><span class="cl">ps aux
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Look at environment variables</span>
</span></span><span class="line"><span class="cl">env <span class="p">|</span> sort
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Test network connectivity</span>
</span></span><span class="line"><span class="cl">wget -qO- http://other-service:3000/health
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Check disk usage</span>
</span></span><span class="line"><span class="cl">df -h</span></span></code></pre></div>
</div>
<p>But what if the container has no shell? Minimal images built <code>FROM scratch</code> or distroless images strip everything out.</p>
<h2 id="docker-logs">docker logs<a class="anchor" href="#docker-logs" aria-label="Link to “docker logs”">#</a></h2>
<p>Before exec-ing into anything, check the logs:</p>
<div class="code">
<div class="code-head"><span class="code-lang">bash</span><button type="button" class="code-copy" data-copy-code hidden><svg viewBox="0 0 24 24" aria-hidden="true"><rect x="8" y="8" width="12" height="12" rx="2.5"/><path d="M16 8V6.5A2.5 2.5 0 0 0 13.5 4h-7A2.5 2.5 0 0 0 4 6.5v7A2.5 2.5 0 0 0 6.5 16H8"/></svg><span>Copy</span></button></div>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># Last 100 lines</span>
</span></span><span class="line"><span class="cl">docker logs --tail <span class="m">100</span> mycontainer
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Follow in real time</span>
</span></span><span class="line"><span class="cl">docker logs -f mycontainer
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Logs since a specific time</span>
</span></span><span class="line"><span class="cl">docker logs --since 2026-03-29T10:00:00 mycontainer
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># With timestamps</span>
</span></span><span class="line"><span class="cl">docker logs -t mycontainer</span></span></code></pre></div>
</div>
<p>The <code>--since</code> flag is underrated. When a container has been running for weeks, you do not want to scroll through megabytes of old logs.</p>
<h2 id="docker-inspect">docker inspect<a class="anchor" href="#docker-inspect" aria-label="Link to “docker inspect”">#</a></h2>
<p>This tells you everything about the container&rsquo;s configuration:</p>
<div class="code">
<div class="code-head"><span class="code-lang">bash</span><button type="button" class="code-copy" data-copy-code hidden><svg viewBox="0 0 24 24" aria-hidden="true"><rect x="8" y="8" width="12" height="12" rx="2.5"/><path d="M16 8V6.5A2.5 2.5 0 0 0 13.5 4h-7A2.5 2.5 0 0 0 4 6.5v7A2.5 2.5 0 0 0 6.5 16H8"/></svg><span>Copy</span></button></div>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># Full JSON dump</span>
</span></span><span class="line"><span class="cl">docker inspect mycontainer
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Just the IP address</span>
</span></span><span class="line"><span class="cl">docker inspect -f <span class="s1">&#39;{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}&#39;</span> mycontainer
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Environment variables</span>
</span></span><span class="line"><span class="cl">docker inspect -f <span class="s1">&#39;{{json .Config.Env}}&#39;</span> mycontainer <span class="p">|</span> jq .
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Mount points</span>
</span></span><span class="line"><span class="cl">docker inspect -f <span class="s1">&#39;{{json .Mounts}}&#39;</span> mycontainer <span class="p">|</span> jq .
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Port mappings</span>
</span></span><span class="line"><span class="cl">docker inspect -f <span class="s1">&#39;{{json .NetworkSettings.Ports}}&#39;</span> mycontainer <span class="p">|</span> jq .
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Health check status</span>
</span></span><span class="line"><span class="cl">docker inspect -f <span class="s1">&#39;{{json .State.Health}}&#39;</span> mycontainer <span class="p">|</span> jq .</span></span></code></pre></div>
</div>
<h2 id="docker-cp">docker cp<a class="anchor" href="#docker-cp" aria-label="Link to “docker cp”">#</a></h2>
<p>Copy files in or out of a running container:</p>
<div class="code">
<div class="code-head"><span class="code-lang">bash</span><button type="button" class="code-copy" data-copy-code hidden><svg viewBox="0 0 24 24" aria-hidden="true"><rect x="8" y="8" width="12" height="12" rx="2.5"/><path d="M16 8V6.5A2.5 2.5 0 0 0 13.5 4h-7A2.5 2.5 0 0 0 4 6.5v7A2.5 2.5 0 0 0 6.5 16H8"/></svg><span>Copy</span></button></div>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># Copy a config file out for inspection</span>
</span></span><span class="line"><span class="cl">docker cp mycontainer:/etc/nginx/nginx.conf ./nginx.conf
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Copy a fix in</span>
</span></span><span class="line"><span class="cl">docker cp ./fixed-config.json mycontainer:/app/config.json</span></span></code></pre></div>
</div>
<p>This works even if the container has no shell or text editor.</p>
<h2 id="nsenter-the-nuclear-option">nsenter: The Nuclear Option<a class="anchor" href="#nsenter-the-nuclear-option" aria-label="Link to “nsenter: The Nuclear Option”">#</a></h2>
<p>When <code>docker exec</code> is not enough or the container has no shell, <code>nsenter</code> lets you enter the container&rsquo;s namespaces from the host:</p>
<div class="code">
<div class="code-head"><span class="code-lang">bash</span><button type="button" class="code-copy" data-copy-code hidden><svg viewBox="0 0 24 24" aria-hidden="true"><rect x="8" y="8" width="12" height="12" rx="2.5"/><path d="M16 8V6.5A2.5 2.5 0 0 0 13.5 4h-7A2.5 2.5 0 0 0 4 6.5v7A2.5 2.5 0 0 0 6.5 16H8"/></svg><span>Copy</span></button></div>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># Get the container&#39;s PID</span>
</span></span><span class="line"><span class="cl"><span class="nv">PID</span><span class="o">=</span><span class="k">$(</span>docker inspect -f <span class="s1">&#39;{{.State.Pid}}&#39;</span> mycontainer<span class="k">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Enter its network namespace</span>
</span></span><span class="line"><span class="cl">sudo nsenter -t <span class="nv">$PID</span> -n ss -tlnp
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Enter all namespaces (like exec but from the host)</span>
</span></span><span class="line"><span class="cl">sudo nsenter -t <span class="nv">$PID</span> -m -u -i -n -p -- /bin/sh</span></span></code></pre></div>
</div>
<p>This is useful for network debugging. You can run <code>tcpdump</code>, <code>ss</code>, or <code>ip</code> inside the container&rsquo;s network namespace even if those tools are not installed in the container.</p>
<h2 id="docker-diff">docker diff<a class="anchor" href="#docker-diff" aria-label="Link to “docker diff”">#</a></h2>
<p>See what files changed since the container started:</p>
<div class="code">
<div class="code-head"><span class="code-lang">bash</span><button type="button" class="code-copy" data-copy-code hidden><svg viewBox="0 0 24 24" aria-hidden="true"><rect x="8" y="8" width="12" height="12" rx="2.5"/><path d="M16 8V6.5A2.5 2.5 0 0 0 13.5 4h-7A2.5 2.5 0 0 0 4 6.5v7A2.5 2.5 0 0 0 6.5 16H8"/></svg><span>Copy</span></button></div>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">docker diff mycontainer</span></span></code></pre></div>
</div>
<p>Output looks like:</p>
<div class="code is-plain">
<div class="code-head"><button type="button" class="code-copy" data-copy-code hidden><svg viewBox="0 0 24 24" aria-hidden="true"><rect x="8" y="8" width="12" height="12" rx="2.5"/><path d="M16 8V6.5A2.5 2.5 0 0 0 13.5 4h-7A2.5 2.5 0 0 0 4 6.5v7A2.5 2.5 0 0 0 6.5 16H8"/></svg><span>Copy</span></button></div>
<pre tabindex="0"><code>C /app
A /app/data/cache.db
C /tmp
A /tmp/something-unexpected</code></pre>
</div>
<p><code>A</code> means added, <code>C</code> means changed, <code>D</code> means deleted. This is great for finding containers that are writing to unexpected places.</p>
<h2 id="debugging-network-issues">Debugging Network Issues<a class="anchor" href="#debugging-network-issues" aria-label="Link to “Debugging Network Issues”">#</a></h2>
<p>When containers cannot talk to each other:</p>
<div class="code">
<div class="code-head"><span class="code-lang">bash</span><button type="button" class="code-copy" data-copy-code hidden><svg viewBox="0 0 24 24" aria-hidden="true"><rect x="8" y="8" width="12" height="12" rx="2.5"/><path d="M16 8V6.5A2.5 2.5 0 0 0 13.5 4h-7A2.5 2.5 0 0 0 4 6.5v7A2.5 2.5 0 0 0 6.5 16H8"/></svg><span>Copy</span></button></div>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># Check which networks a container is on</span>
</span></span><span class="line"><span class="cl">docker inspect mycontainer -f <span class="s1">&#39;{{json .NetworkSettings.Networks}}&#39;</span> <span class="p">|</span> jq <span class="s1">&#39;keys&#39;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Ping from one container to another</span>
</span></span><span class="line"><span class="cl">docker <span class="nb">exec</span> web ping -c <span class="m">3</span> api
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Check DNS resolution</span>
</span></span><span class="line"><span class="cl">docker <span class="nb">exec</span> web nslookup api
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># List all containers on a network</span>
</span></span><span class="line"><span class="cl">docker network inspect mynetwork -f <span class="s1">&#39;{{range .Containers}}{{.Name}} {{.IPv4Address}}{{end}}&#39;</span></span></span></code></pre></div>
</div>
<h2 id="one-liner-health-checks">One-Liner Health Checks<a class="anchor" href="#one-liner-health-checks" aria-label="Link to “One-Liner Health Checks”">#</a></h2>
<div class="code">
<div class="code-head"><span class="code-lang">bash</span><button type="button" class="code-copy" data-copy-code hidden><svg viewBox="0 0 24 24" aria-hidden="true"><rect x="8" y="8" width="12" height="12" rx="2.5"/><path d="M16 8V6.5A2.5 2.5 0 0 0 13.5 4h-7A2.5 2.5 0 0 0 4 6.5v7A2.5 2.5 0 0 0 6.5 16H8"/></svg><span>Copy</span></button></div>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># Which containers are unhealthy?</span>
</span></span><span class="line"><span class="cl">docker ps --filter <span class="nv">health</span><span class="o">=</span>unhealthy --format <span class="s1">&#39;{{.Names}}: {{.Status}}&#39;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Which containers restarted recently?</span>
</span></span><span class="line"><span class="cl">docker ps --format <span class="s1">&#39;{{.Names}}: {{.Status}}&#39;</span> <span class="p">|</span> grep -i restarting
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Resource usage snapshot</span>
</span></span><span class="line"><span class="cl">docker stats --no-stream --format <span class="s1">&#39;table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}&#39;</span></span></span></code></pre></div>
</div>
<p>The instinct to restart a broken container is strong. But nine times out of ten, the information you need to fix the actual problem is sitting right there in the logs, the inspect output, or the filesystem diff. Restarting just resets the evidence.</p>
]]></content:encoded></item></channel></rss>