Stumbled across this gem from nachash. Interesting read on over two
years of the purest freedom of speech you can find and the effort it
took to maintain.
[tor-dev] yes hello, internet supervillain here
Fears No One nachash at observers.net
Sat Nov 8 22:10:23 UTC 2014
Previous message: [tor-dev] [PATCH] Pinning middle nodes for HSes: anti-guard-discovery
Next message: [tor-dev] yes hello, internet supervillain here
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Sorry in advance for the length. I just want to make sure that
everything is included. If you have any questions/clarifications, just
ask. It isn't every day that someone like me pops up on tor-dev, and I
just want to make sure this is as productive and helpful as possible.
This will probably be a very humbling experience, because unlike my
fellow illegal onion operators both past and present, I will actually be
outside of a jail cell and able to read the ruthless dissection of my
set-up. On the bright side, you're all are getting way more info from me
than the pigs will ever willingly cough up, which means if they have
some sort of magic onion decloak trick, this mailing list discussion is
a good chance at finding it. All of these files are in the hands of the
cops anyway (And I have no plans of bringing doxbin back), so there are
0 real-time opsec concerns.
First, the files:
sha256 and sha512 checksums for all the files are at the bottom.
NOTE: This isn't my box and I didn't set it up.
WARNING: The .xz files will unpack to roughly 1 GB.
Some other info:
1. The box (An OpenVZ VPS) was hosted with Hetzner in Germany. People on
Twitter keep asking if the box was in Bulgaria, but we didn't use
Bulgaria for one simple reason: The very first doxbin box (Bought in
2011) was with a VPS company in Bulgaria. After the first month, they
said "TOR IS ILLEGAL" (I shit you not), killed the box, and kept the $5
we had paid for the 2nd month. Can't say I blame them.
2. It wasn't transproxied. We did this once before, but it became a
hassle to drop iptables rules just to upgrade tor, especially when tor
was getting regular updates semi-recently. Console access with VPS
companies generally requires java plugins or something just as gross, so
there aren't a lot of sane options here. It gets annoying when there's a
surge in the frequency of tor upgrades, so we stopped transproxying the
box. inb4 OMG OPSEC MISTAKE DETECTED!!!11
3. This script (And copypasta that contained its important parts) was
used to build nginx for the various boxes from roughly November 2011 to
mid-October 2014: http://pastebin.com/dBC7E8Jd Early versions didn't use
naxsi, and sed replaced the server banners in the source before
compiling. Yes, I know it doesn't verify the source. See the next point:
4. Around 2 weeks ago, I started grabbing dotdeb.org's source versions
of nginx and building .debs using hardening-wrapper with the following
command at the end:
DEB_CFLAGS_SET="-O2 -fstack-protector-all -fforce-addr -ffast-math
-fomit-frame-pointer -falign-functions=64 -falign-loops=32"
dpkg-buildpackage -uc -us -j8
5. The tor build was from deb.torproject.org. I'm not able to check the
box and never thought to back up /etc/apt/sources, but it used the
experimental-wheezy repo. All updates were done within 24 hours of the
new .debs going live.
6. Last but not least, the elephant in the room: The php is a
headache-inducing nightmare. I inherited the code and it worked, so I
just papered over its defects over the years (There was no anti-flood
protection originally, for example) and built it up to resemble Kowloon
Walled City expressed in php. I have a feeling that a lot of focus is
going to be placed on the code, to the detriment of finding any possible
tor bugs. In any case, everything about doxbin's setup is being
disclosed in the spirit of making this an interesting learning
experience for all parties involved.
Upon scrolling through the .xz files (I personally use xzless), you'll
find a bunch of stuff like:
All of the requests were around (If I recall correctly) 3KB in size.
Oddly enough, it caused tor to hiccup pretty badly, although the web
server itself was just fine and I didn't have any network bandwidth
problems (i.e. No obscene traffic spikes). It's also worth pointing out
that the tcp buffers weren't even close to maxed. The same box has
handled a similar volume of legitimate requests before (Namely back in
March, after The Hidden Wiki debacle; see
https://twitter.com/loldoxbin/status/530765088366821377). The solution
to getting tor availability back was to set ConstrainedSockets to 1 and
play with ConstrainedSockSize (Originally set to 8192, then 4096). This
made doxbin regularly available again, whereas before it was hit or
miss. Once the requests stopped, I waited a couple of days before
commenting those two config lines out and reloaded tor.
A month later, the same kinds of requests started coming in again. After
the first few hours, I started 301 redirecting all requests containing
/%5C%22 to the new Hidden Wiki's Hard Candy page. I also added a grep -v
to my log report script in order to filter out the noise (Possibly a
mistake, but we both tailed logs and watched for something like a
different attack style that the ddos was being used to cover and never
noticed anything). That was good enough to maintain availability, so I
rolled with it and the requests eventually stopped. I have no hard data
on that last point, just the fact that I tailed the access log and the
requests went from 5 per second to 1 every 3-6 seconds before dying off
I said on Twitter that we suspected it was a deanonymization attempt,
but I didn't elaborate why because LOL 140 CHARACTERS. Intangir (The
other admin, who took over for me from Halloween 2013 to some time
around July of this year) and I talked about it back in August and
decided that an attacker was probably involved spraying specially
crafted packets at the box in order to mess up its circuits, and
eventually get us on attacker-controlled nodes. Since we mitigated the
availability loss, we deemed it as no big deal. In hindsight, that seems
hilariously stupid/naive of us. The kid who started doxbin had a similar
theory that I'm just going to paste verbatim:
<founder> i think
<founder> the attack
<founder> was to DoS you
<founder> until you created circuits
<founder> entirely made of dickbleedable nodes
<founder> and then dickbleeding them
<nachash> but the server
<nachash> got seized
<founder> yeah, the IP was discovered by dickbleed though
<founder> the entire circuit was leaking info
<nachash> lol, did you just reproduce this?
<founder> not yet, i'll be trying
<nachash> Do you mind if I share this with tor devs?
<founder> go ahead
<founder> its just a theory at the moment
If either of these theories even remotely pan out, a possible mitigation
for the next person like me (Which shouldn't require any tor dev work)
might look something like this:
1. Make several public relays and configure the torrc of the hidden
service to only make circuits which begin with those nodes. Private
bridges would be a liability in this case because anyone who figures out
the guard node by weaponizing one of attacks from a whitepaper (Which
guard nodes were modded to mitigate) will be able to find the guard node
and then quickly discover that it isn't part of the public tor network.
Public bridges might be ok, but probably less so.
2. Cross your fingers and pray really, really hard that the money trail
is correctly obscured.
4. PROFIT! (Or lulz, in my case)
(No, the similarity between the idea of drug market operators giving
back to the network by donating nodes to the practice of drug cartels
building schools and hospitals is not lost on me)
Another thing to consider is the recommendation against running a hidden
service as a relay. Of course, the argument against doing so in the
documentation is very sound. At the same time, the FBI has stated in the
Silk Road criminal complaint that upon finding the IP, the investigator
verified that it wasn't a tor node and knew he had won the onion
lottery. Of course, that could still be psyops/parallel construction
garbage to scare people into making their jobs a little easier, but it's
something to thing about.
sha256sums of all the files:
Hope this helps,