Hacked!

With our school’s graduation ceremony last night, the school year is now officially finished. This year will definitely go down in my memory as the year that the students got the best of me… twice!

IP-gate

To give some background on the first “hack,” our current network uses a flat IP network with IP subnets used for each different set of machines (for organizational purposes). We don’t use IP-based security for obvious reasons, but we do use the subnets for deciding internet speed. IP addresses (fixed except for the guest subnet) are given out using DHCP, and each of the subnets except the guest subnet gets decent speed.

When I set up this system ten years ago, I was well aware of the obvious drawback: any person could set a static IP address on any subnet they chose, and, given our lack of managed switches (at the time we had none, though things are changing), there wasn’t much of anything I could do about it. On the flip side, the worst that could happen is that these users would get faster internet, hardly the end of the world.

It took ten years, but, finally, someone figured it out. One of our more intelligent students decided that his IP address of 10.10.10.113 didn’t make a whole lot of sense, given that the gateway is 10.10.1.1. He set his IP to 10.10.1.113, and, voilà, his internet speed shot through the roof!

Naturally, he shared his findings with his friends, who managed to keep it under the radar until one of the friends decided to see how well BitTorrent would work with the school internet. What none of these students realized is that the 10.10.1.* subnet was for servers, and, oddly enough, none of our servers uses BitTorrent. The traffic stuck out like a sore thumb, and I finally caught on.

My first step was to blacklist all unrecognized MAC addresses using the server subnet. The next step was more difficult. Now that the cat was out of the bag and everyone knew how to get faster internet, I needed a way to block anybody not using the IP they’d been assigned through our DHCP server. Obviously, there is a correct way of doing this, but that seems to be using 802.1x, and we’re just not there yet. My quick and dirty solution was to copy the dhcp configuration file containing all the host and IP information to our firewall, and then generate a list of iptables rules that only allow traffic through if the IP address matches the expected MAC address.

The problem with this solution is that it doesn’t account for the fact that spoofing MAC addresses is actually relatively simple, so it looks like one of my summer projects is going to be a complete revamp of our network. I’m hoping I can configure our FreeIPA server to also operate as the backend for a RADIUS server so we can implement 802.1x security.

In this case, the consequences of the “hack” for us were pretty insignificant. Students got some extra bandwidth for a while. The students who changed their IP addresses also didn’t suffer any major consequences. Their devices were blacklisted from the internet until they came to speak with me, and then were put on the guest subnet. All of the students were in their final year, so they were only stuck on the guest subnet for the last month or so.

The most obvious lesson I learned from “IP-gate” is that security through obscurity works great…until someone turns on the light. And when that happens, you’d better have a plan.

The Grade-changing Scandal

This was a far messier situation. One of our teachers allowed a student to access their computer to set up a video for class. On the computer, the teacher had saved their login credentials for LESSON, our web-based marking system. While the teacher was distracted, the student used this trick to find the teacher’s password, and then shared the password with different members of the class. Throughout the next few days, the class average for that teacher’s subjects rose at a remarkable rate.

Three days later, one of the students finally told the principal what had happened, and the principal called me. What followed was a day of tying together evidence from multiple sources to work out who changed what and when.

What the students weren’t aware of was that LESSON logs everything at the assignment level, so I could see which IP addresses changed which assignments. If the IP was an internal school address, I could also see which user changed the assignment. One of the students used their laptop (registered on the network, so I knew who it was) to change some marks, then logged in from a lab computer (so once again, I knew who it was), and then finally logged in from home.

The students who logged in from home were harder to track, at least until they did something foolish, like logging in as themselves to verify that the marks had actually changed ten seconds after logging out as the teacher.

We also do daily backups of the LESSON database that we keep for a full year, so it was a piece of cake to restore all of the marks back to their original scores.

Obviously though, this went much further than the IP-spoofing going in in “IP-gate.” This wasn’t just some kids wanting faster internet, this was a case of flagrant academic dishonesty.

In the end, we came up with the following consequences:

  • The students who masterminded the break-in received a zero for the subject for the term
  • The students who we caught changing the marks received zeros for any assignment of theirs that had a changed mark
  • The students we knew that they knew their marks were changed received three Saturday detentions (they have to sit in complete silence for four hours on a Saturday)
  • The students we suspected that they knew their marks were changed received one Saturday detention, though these students were allowed to appeal, and most who did had their Saturday detention reversed

One of the things I’ve learned from this is that there’s never too much audit information. LESSON is going to be changed to record not just who changes each assignment, but who changes each mark, and there will be a history of every changed mark so that teachers can see when marks are changed.

Apart from this, I would be curious as to what others think about the consequences for these two “hacks.” Were we too lenient on the first? Too harsh on the second? What should we have done differently? And what should we do differently going forward?

Laptop computer locked with chain and padlock by Santeri Viinamäki. Used under a CC BY-SA 4.0 license.

Finishing the LESSON

At our school, we’ve been using a web-based marking system called LESSON for the last twelve years or so. I originally wrote LESSON because I was annoyed with the complexity of using a spreadsheet to deal with both assignments of different weights and absences. It started out as a personal web-based gradebook, written in PHP with all the compromises that implies. I didn’t use classes, created only a small number of core functions, and made liberal use of copy and paste. My code was a mess of spaghetti that only got worse as time went on. In other words, I wrote a typical PHP website.

During the final half of 2011, I went home to the States on sabbatical, and, when I returned in early 2012, my teaching load was less than normal for the remainder of the school year. One of my goals for the year was a rewrite of LESSON. I had a vision of LESSON 2.0 having a backend written in Python, a well-written web frontend in PHP or Python and an Android client. Using sqlalchemy and web.py, I came up with a core backend that automatically generated pages based on the database tables, allowed filtering using URLs, and allowed modules to override the automatically generated pages when necessary. The code was elegant (compared to the old LESSON 1.0 code), resilient and much easier to extend.

Unfortunately, other projects started to take my attention off of the backend, and though I was happy with the core, I never reached the point of even starting to port our current system to the backend. The backend has languished since mid-2012 (coincidentally, when I started teaching my usual load again). There was a small resurgence of commits last October, when I had great intentions of resurrecting the project in preparation for some major work integrating family information, but that rapidly tapered off as other projects took precedence.

Compare that to LESSON 1.0 which has seen more or less continuous development over the last twelve years. This year, I’ve mainly focused on moving our school registration process over to LESSON, and, more recently, Telegram integration, but it’s telling that all that work has been done on LESSON 1.0.

So why does the “old” LESSON get all the updates? I think it’s for the following reasons:

  • LESSON 1.0 works and works well. Its code might be a nightmare, but it’s fast, low-bandwidth and easy to use, even for teachers who have never touched a computer before (and we had a few of those when we first started using LESSON). LESSON has a grand total of two images. Last month, we had 1,009,000 external hits on LESSON with a total bandwidth of less than 2GB. That’s under 2KB per hit. And, despite multiple student attempts to convince me otherwise, there hasn’t been a bug in LESSON’s grading code in years.
  • New features trump better code. I am a system administrator, teacher and programmer. Our day-to-day sysadmin emergencies always need to be dealt with before long-term projects and my teaching is definitely next on the list of priorities. When I do have time to work on long-term projects (maybe 10-15 hours a week), new features always have the highest priority. The principal wants to know when he’ll be able to create a Telegram channel, populate it with all of his teachers, and send them a message. He’s never asked me when I’ll be done with LESSON 2.0.
  • I fell victim to the second system effect. When developing LESSON 2.0, I spent a lot of time trying to create a system rather than a program. LESSON 1.0 wasn’t designed to be extensible, but LESSON 2.0 was, complete with the concept of modules and all sorts of other goodies. Unfortunately, these added a level of complexity that made the whole thing more and more difficult to work with.

So where does this leave us? The code for LESSON 1.0 may be ugly, but it works well enough. If I ever get caught up with everything else, I’ll probably continue working on LESSON 2.0, but for now it’s on the back burner, and, like anything left on the back burner for too long, it may eventually get thrown out.