FIG Revokes Chinese Gymnastics Medal

I Did My Best

When I meet people in computer security circles or at conferences and I'm identified as Stryde, the Olympic story comes up pretty often, and the comment I seem to get the most is "how did you let them get away with it?" I've never understood this sentiment, the idea that I somehow have more power than a Publish button on Blogger, and I often counter with "Well I was going to launch an airstrike but I reconsidered." Little did I know, the president of the International Gymnastics Federation (FIG), Bruno Grandi, has been using the same joke.

From This interview:
"Grandi said it was conceivable that China had cheated in Beijing"
"There was strong circumstantial evidence, certainly, but these investigations are not my job ... I'm not the police or Interpol. If I find that there was cheating, then I can act." ... "I had everything sent to the IOC and the IOC has carried out its investigations and the figures were the same ... The IOC gave us its findings, and we checked them and there was nothing. When people on the Internet find fake documents, you need to legally prove that these are fake, and that's not my job. I have to respect the documents that the Chinese government gives me. What else should I do - declare war on China?"

Putting aside for a moment the "fake documents" phrase, and assuming for a moment that Grandi does not understand that the documents I linked to were hosted by the Chinese government itself, verifiably, for years, the most important part of that interview is the last phrase. Grandi's process, and the IOC's process, requires them to trust documents that are provided to them by governments. This is a great process for finding athletes that are cheating, and a totally failed process for finding governments that are cheating. What justice system would make the defendant an authority on his own guilt? Only the IOC and the FIG.

As Grandi says, a fact-finding authority with the power to prosecute a government for fraud was never involved. The servers from which the world watched the Chinese government censor the truth in real time were never seized and forensically analyzed. No one was ever caught or prosecuted for deleting any of the primary documents off of Chinese government web servers as age records vanished one by one under our watching eyes. And the reason for this is that the FIG by their own admission are not empowered to question governments. So when we say that the Chinese gymnasts were cleared by the FIG, we need to be very clear about what that actually means: not much.

A Confession Is Not Enough

About a week after the translated Yang Yun video created by myself and Heather Lawver was posted, the FIG re-opened their investigation into Chinese gymnasts competing in the Sydney Olympics. Yesterday, as an alert reader pointed out, the FIG revoked the Bronze medal awarded to Dong Fangxiao in Sydney, due to paper evidence they managed to find of her employment under her real age. The Chinese government immediately responded, claiming "there is no problem in Dong Fangxiao's age." What's interesting is that the FIG did not revoke the medal of Yang Yun, who is seen here confessing on state television to competing under age. It is important to note that for the FIG, a videotaped confession was not sufficient evidence. So, to summarize:

• The FIG is by their own admission not authorized to investigate governments for fraud


• The documents I identified implicate the government as having committed fraud


• The removal of every single linked document from government web servers indicates fraud


• A videotaped confession is considered insufficient proof


• Despite all these limitations, the Chinese have still lost a medal for age falsification

Readers are welcome to examine my archives and what evidence is left and make their own conclusions. I haven't written about this since it happened because I felt it became a sports story and not a technology story, and the technology story I was interested in was becoming lost in the noise. I wanted to talk about document permanence, transparency, and the amazing impact both were having on our culture. Fox News just wanted me to say that someone was cheating. In the end, as the FIG prepares to take a medal back, I will say only this: I stand by the integrity of my findings.

- stryde.hax

Network Fingerprint for LANRev Agent

Analysis of the LANRev software used to spy at Harriton High continues; for now, I wanted to give everyone a way to detect if a machine has the software installed this won't tell if its webcam features are active. You'll need a Unix command line with the "netcat" utility installed. Then, paste in the command below, but use the IP address of the computer you'd like to check instead of the one in the example (10.0.7.22).

/bin/echo -e "\\x01\\x00\\x03\\x00\\x00\\x00\\x00\\x00
\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff
\\xff\\xff\\xff" | nc -v -v -v  10.0.7.22 3970

strydes-Mac-mini.local [10.0.1.111] 3970 (?) open
00000000  01 00 00 03 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000010  00 00 01

Above, you can see the 19 bytes you will receive in response if the computer in question is has LANRev. Additional bytes will follow, up to 272 total bytes, but those are subject to encryption so I cannot predict them. The above fingerprint is enough. I've submitted a fingerprint to the Nmap fingerprint database, but until that goes through, the above should suffice. Good luck!
-stryde.hax

update 2/28/2010 10:15am

Is it spyware? Well, [Sunbelt AntiVirus] [Philly.com] [New Jersey Star-Ledger] [Pueblo Chieftan] [Cory Doctorow] [Karl Denninger] [Mark Kosur].

update

LANRev says they're removing the webcam functionality, so in response, I've made some edits to this post. I want to be make sure folks know that this test doesn't verify webcam activation functionality.

The Spy at Harriton High

This investigation into the remote spying allegedly being conducted against students at Lower Merion represents an attempt to find proof of spying and a look into the toolchain used to accomplish spying. Taking a look at the LMSD Staff List, Mike Perbix is listed as a Network Tech at LMSD. Mr. Perbix has a large online web forum footprint as well as a personal blog, and a lot of his posts, attributed to his role at Lower Merion, provide insight into the tools, methods, and capabilities deployed against students at LMSD. Of the three network techs employed at LMSD, Mr. Perbix appears to have been the mastermind behind a massive, highly effective digital panopticon.

PanoMasterMind

The primary piece of evidence, already being reported on by a Fox affiliate, is this amazing promotional webcast for a remote monitoring product named LANRev. In it, Mike Perbix identifies himself as a high school network tech, and then speaks at length about using the track-and-monitor features of LanRev to take surreptitious remote pictures through a high school laptop webcam. A note of particular pride is evident in his voice when he talks about finding a way outside of LANRev to enable "curtain mode", a special remote administration mode that makes remote control of a laptop invisible to the victim. Listen at 35:47, when he says:

"you're controlling someone's machine, you don't want them to know what you're doing"
-Mike Perbix

It isn't until 37 minutes into the video till Perbix begins talking about the Theft Tracking feature, which causes the laptop to go into a mode where it beacons its location and silent webcam screenshots out to an Internet server controlled by the school.

Click to watch an excerpt of Mike Perbix's spycast

The beacon feature appears to have been one of the primary methods for remote spying, however, network footprints abound over the details and architecture of the remote administration effort. In this post, Perbix discusses methods for remotely resetting the firmware lockout used to prevent jailbreaking of student laptops. A jailbreak would have allowed students to monitor their own webcam to determine if administrators were truly taking pictures or if, as the school administration claimed, the blinking webcams were just "a glitch."

Perbix also maintains a prolific blog, where in this blog post he describes using the remote monitoring feature to locate a stolen laptop:

"As a prime example, we initially attempted to recover a stolen laptop that reported back to us it's internet address and DNS name. The police went to the house and were befuddled to find out the people we knew had the laptop was not the family that lived there...well, we eventually found out that they were the neighboring house and were borrowing the unsecured WI-FI."

In a September 2009 post that may come to haunt this investigation, Perbix posted a scripting method for remote enable/disable of the iSight camera in the laptops. This post makes a lot more sense when Perbix puts it in context on an admin newsgroup, in a post which makes it clear that his script allows for the camera to appear shut down to user applications such as Photo Booth but still function via remote administration:

"what this does is prevent internal use of the iSight, but some utilities might still work (for instance an external application using it for Theft tracking"

What's the purpose of shutting down a camera for the user of the laptop but still making it available to network administrators? Ask yourself: if you wanted to convince someone that a webcam blinking was a glitch, would disabling the cameras help make your case?

We Found the Glitch, Mrs. Buttle

The truly amazing part of this story is what's coming out from comments from the students themselves. Some of the interesting points:

• Possession of a monitored Macbook was required for classes


• Possession of an unmonitored personal computer was forbidden and would be confiscated


• Disabling the camera was impossible


• Jailbreaking a school laptop in order to secure it or monitor it against intrusion was an offense which merited expulsion

When I spoke at MIT about the wealth of electronic evidence I came across regarding Chinese gymnasts, I used the phrase "compulsory transparency". I never thought I would be using the phrase to describe America, especially so soon, but that appears to be exactly the case. On a familiar note, the authorities are denying everything. As one reads comments on this story, a consistent story begins to emerge:

"My name is Manuel Tebas. I was a student at Harriton High School, in the graduating class of 2009. We were the first year on the one-to-one laptop initiative. [...] I saw your post about removing webcam capability from the Macbook. It is possible - I did it last year. I will preface this by saying that when I did it, I was almost expelled, saved only by the fact that there was, at the time, no rule against doing so."

"I remember that the laptop was a requirement in school for many classes. That may remain so."

" had brought in my own personal computer to work on a project for school one day. I was doing a presentation involving programs not available on the regular computers, only in specific labs. I happened to have a copy of my own. My personal property was confiscated from me in a study hall when I was working on a school assignment because it was against the schools 'code of conduct'."

"Hi, I'm a 2009 Graduate of Harriton Highschool. [...] I and a few of my fellow peers were suspicious of this sort of activity when we first received the laptops. The light next to the web cam would randomly come on, whether we were in class, in study hall or at home minding our own business. We reported it multiple times, each time getting the response: "It's only a malfunction. if you'd like we'll look into it and give you a loaner computer."


"The webcam couldn't be disabled due through tough tough security settings. Occasionally we would notice that the green light was on from time to time but we just figured that it was glitching out as some macbooks do sometimes. Some few covered it up with tape and post its because they thought the IT guys were watching them. I always thought they were crazy and that the district, one of the more respectable ones within the state, would never pull some shit like this. I guess I was wrong."

"I am the father of a 17 y/o Harrington High student. She has had one of these laptops for 2 years. She has noticed the "green light" coming on but was not computer literate enough to know what initiated it"

Browse as many web forums as you like, the comments above are highly representative. Students were told green webcam activation lights going off at home were a glitch, were required to use a jailed computer, were threatened with expulsion if they attempted to jailbreak the computer to find the truth, and were not allowed to use computers they controlled.

Inside LANRev

With some of my colleagues, I began a reverse engineering effort against LANRev in order to determine the nature of the threat and possible countermeasures. Some of the things we found at first left us aghast as security pros: the spyware "client" (they call it an agent) binds to the server permanently without using authentication or key distribution. Find an unbound agent on your network with Bonjour, click on it, you own it. The server software, with an externally facing Internet port... runs as root. I'm not kidding. For those unfamiliar with the principle of least privilege- this is an indicator of a highly unskilled design. Unfortunately, when we got down to basic forensics, LANRev appears to cover its tracks well. Here's a screenshot of the server application monitoring a tracked host:

Tracking intervals available at the top; screenshots and webcam shots in the lower right pane. No webcam shot is visible here as a webcam was not connected during testing

In order to spy on my computer, I had to mark it for spying. The icon for spying is a detective hat and a magnifying glass; very Sherlock Holmes

Once I had the agent installed, I used dtrace to monitor its activity as it hung around and spied on my system. The log below is an edited trace of the agents activity during a spy interval. It uses a fixed dump point, /tmp/Image, as its save file before uploading to the server, sadly this is wiped. Only a full forensics scan which picks up deleted files will have a chance of picking up the history of the spying on a particular computer. On laptops with a webcam, a second fixed save point, /tmp/Image1, is used to save the webcam pic.

For the technically inclined, I've highlighted some of the key points, use of the system screengrabber, the use of RawCamera, the fixed save point, etc. We're still working on our technical writeup of this software and hope to update soon.

During our testing, we infected a laptop with LANRev, then closed the lid, hoping to activate the LANRev feature which takes a webcam picture when the computer wakes. As my colleague Aaron opened the lid of his Mac, the green webcam light flickered, ever so briefly. It wasn't a glitch. It was a highly sophisticated remote spy in his system. And even though he was in control, the effect was still very creepy.

Here's one last capture from the Windows version of the administration console, showing a forced remote webcam snapshot. We've pixellated this, but rest assured the real thing looks very detailed

In other news on the case, subpoenas have been issued, the FBI is on the case, the candy in question has been caught red-fingered, and some enterprising chap is ready to cash in with a t-shirt. Doug Muth's hands on screenshots provide the best first hand encounter with the client end of the spyware in question. What amazes me most is that the family and lawyer filing the suit appear to have done no digital forensics going in, and no enterprising student hacker ever jailbroke a laptop and proved this was going on. The greatest threat to this investigation now is the possibility that the highly trained technical staff at LMSD could issue a LANRev script to wipe digital forensic evidence off all the laptops. This is why it is imperative for affected parents to have the hard drive removed from their children's laptops and digitally imaged before the laptop is connected to a network. With enough persistence, and enough luck, we may eventually learn the truth.

-stryde.hax

---

update 3/31/2010 8:00pm

Harriton is just the tip of the iceberg!

update 2/25/2010 11:00pm

I have a question about this case I haven't seen answered to my satisfaction anywhere. I believe it is possible, even likely, that the webcams were only activated on stolen laptops. A persistent contingent of commenters have no problem and see no 4th amendment issue with a state agency, such as a school, activating a webcam remotely to locate a laptop thief. Even if that thief is a child. Even if the thief is a child at home in their bedroom. Even if that thief is changing clothes? Read the argument made here. Comments? Thoughts? Expert feedback?

update 2/23/2010 6:00pm

If you haven't already, you must watch this PBS Documentary - How Google Saved a School. At five minutes in, you can see all these same features in use, in a school setting, by a principal. Remote surreptitious observation. Remote camera use. All used by a principal to observe kids and make sure they're working. There are a lot of school districts, administrators, IT professionals, and security professionals who see nothing wrong with this documentary. They see remote administration software in use in this way and they don't think it's wrong, and they don't think it's spyware. Some of them even believe that the extension of this functionality into the home doesn't make it spyware, or even wrong. But this is my personal blog, and it's my personal opinion that they're wrong. As an expecting parent, I don't ever want my kids on the business end of Remote Desktop Curtain Mode, even at school. I'm a security professional, and a big part of my education and my professional development was tinkering and tearing apart computer systems to gain understanding, learn how they work, and change their use. I believe that computer security is knowledge in practice; it's using your knowledge to protect yourself. These kids are learning that security is something that happens to you. That's backwards. DARPA thinks we're not raising a generation with applicable security skills. I think they're right; I think this is a recipe for the next generation of phishing victims. I'd like to see a school system where a kid can bring in x64 Ubuntu or Haiku OS that he secured him/herself. I'd like to see a school system where kids teach each other how to defend against remote webcam use. Instead, we've got kids who can't run Terminal. Not my kids.

update 2/23/2010 4:12pm

A note for anyone wishing to contact me privately: if you'd like me to write back, please leave a return email. My email is still stryde dot blog at gmail dot com.
It's Not Spyware!!!
I've received a lot of positive feedback about this entry; however, if there's one consistent complaint amongst my detractors, it's my classification of LANRev as spyware. So here is my response. Confusing remote admin software with spyware has a long history stretching back to Cult of the Dead Cow's first Bo2k release. I'm not as funny as them so I don't even try. It's true however that remote administration tools and spyware exist on the same spectrum, just ask the guys at Spectresoft. Spyware authors and remote admin authors often have to solve the same problems, like bypassing OS protections and getting around antivirus. It's a transition that's easily made. So where's the dividing line? The line is basically in how its used. Remote admin usually solves constructive tasks, like remote patch management, inventory location tracking, remote software installation. And sometimes it means screensharing in order to solve problems. I personally have sat at home as a network tech worked on my corporate laptop over a VPN. No problem. My personal opinion is this: when you see a piece of software with dedicated functionality for taking webcam screenshots surreptitiously and removing the evidence on disk, to me that's crossed the line into spyware. I'm certain that others in the industry will disagree with me. That's fine; let's have the debate. I don't mind losing a technical argument, as long as it's on merit.

update 2/23/2010 11:28am

My colleague Aaron pointed out to me today that the reason LANRev is using the raw camera device is that Apple implemented security measures to prevent remote activation of the webcam in OSX. LANRev was designed to bypass this security measure. Those who disagree with my spyware assessment, ask yourself, "what kind of software bypasses OS security measures?"
On the topic of whether or not we yet have proof of illegal use, I would ask you to listen carefully to the webcast, and listen for the word "house" at 1:28. Listen for "yes we have used it."

update 2/23/2010 10:00am

I've removed Mr. Perbix's picture from my blog. I try very hard to stick to verifiable facts when I write here; this blog post is made up references to primary documents that show a verifiable pattern of action. But I feel that some readers are getting carried away. Myself and Aaron Rhodes spent hours reading forum posts, messages, and communications from Mike Perbix, his "digital shadow". The impression we both got was of a man who was charged with enormous responsibility, worked very hard, was very adept, and was fanatical about protecting kids and the assets he was charged with managing. I don't have all the facts yet, but the impression I got was of someone who was trying to build a state of the art capability and revelled in the promise of technology. If I had to put my finger on what when wrong here, I would say that someone cared too much. Personally I'm much more interested in who this capability was distributed to, and its persistent pattern of access, than I am in the person who built it. If you're reading this, please, let us not participate in a rush to judgement especially against a guy who worked this hard. Yes, he built the capability. Yes it was used. But if it was abused or simply misguided, that remains to be proven. I for one reserve judgement. For now, what bothers me most is this: When an organ of the State (in this case, a school) builds a system to conduct a search by activating webcams off of school grounds, the only way to determine if the ensuing search will be unreasonable or illegal is to conduct the search. The thought process behind that is unfathomable to me, no matter how much I read about it.

update 2/22/2010 8:30pm

I've created a network footprinting capability for parents, students, anyone who may be concerned that they are infected with the LANRev agent. The capability is documented in my next blog entry. One piece of feedback I continue to get is speculation on what can be seen in a packet sniffer. The answer for now is: not much. A block cipher and compression are in use in serial. It's a tough problem; we're working on it.

update 2/22/2010 5:30pm

In a strange twist, the makers of LANRev have come out with a statement saying that school network techs should never have used their software to engage in theft recovery:

"We discourage any customer from taking theft recovery into their own hands," said Stephen Midgley, the company's head of marketing, in an interview Monday. "That's best left in the hands of professionals."

I've watched the 50 minute screencast repeatedly, where Perbix describes his use of this feature outside of school grounds repeatedly during a conversation with Absolute Software employees. They were enthusiastic... now they're throwing LMSD under the bus? I believe this can best be described as intense PR spin. It also completely confirms what I've asserted here, that LANRev was the implant of choice for this school.

---

This investigation was conducted by myself and Intrepidus consultant Aaron Rhodes; Aaron deserves credit for a lot of these findings. -*-

The Technology Behind School Spying

Ghost in the Classroom Machine

There's been a flood of news coverage regarding this lawsuit against Harriton High School in Lower Merion, PA. The charge, in short, is that school-issued laptops have been employed by school administrators to surreptitiously webcam-monitor children in their own homes via the use of remote control software, violating nearly every wiretapping statute on the books as well as potentially generating child pornography. While the school has already issued a formal apology regarding the use of remote monitoring technology, this blog post is an attempt to ascertain what technical methods were used to remotely monitor students in their home.

Think Spyware

The first step to identifying the particular spyware in use is to identify the platform. This school document identifies the hardware and software in use as Macbooks running OSX. What's the go-to spy product of choice for school administrators on the OSX platform? Apple Remote Desktop 3:

Possibly the most unfortunate product logo of all time

What proof exists that this product can be used for remote monitoring of students? Remote observation and control of target computers is plainly listed in the Apple Remote Desktop 3 Feature List. But the best evidence is this PBS documentary (about a different school), in which a high school assistant principal is shown listing, monitoring, and remotely taking pictures of high school students using Apple Remote Desktop 3:

Current Status, Current Application, Current User: Apple Remote Desktop

Five minutes, twelve seconds into the video:

"They don't even realize that we are watching. I always like to mess with them and take their picture."
-Assistant Principal Dan Ackerman

This story is only a day old, and if the published numbers are correct, nearly 1800 children and their families may have been exposed. I don't have information yet on what forensic traces this spying may have left on the computers, however, I can recommend best practices for any parent who believes their school system may be using issued hardware to spy on their children (in Lower Merion or elsewhere):

• Understand that most laptops have a microphone and a video camera embedded, and that remote activation of microphones can be utterly silent.


• If the issue becomes public, as is the case above, connecting the laptop to a school administered network or VPN may allow administrators to remove forensic traces of spying. Do not network the computer until evidence collection is complete.


• Seek out a computer security professional or your helpful neighborhood hacker to perform a full forensic hard drive capture of the potential spy platform.


• Consult a lawyer before confronting school officials. Capturing live network forensic evidence of remote spying can be far more powerful than word-of-mouth allegations.

This story is generating a lot of questions in the press, questions about how cameras should be deployed by schools in children's homes, and what guidelines should be set for their use. Personally, I believe these are the wrong questions. I believe the right question is: should students be subject to remote surreptitious monitoring by their school systems at all? Do we want our kids to grow up always wondering who's watching?

---

Big Brother: Remain EXACTLY where you are! Make NO move until you are ordered!
[painting falls from wall, revealing a telescreen]
Julia: Now they can see us...
Big Brother: NOW WE CAN SEE YOU!