URGENT: Probable database breach

SwampFae

Super Moderator
Staff member
Many thanks for halting the attack vector so swiftly.
Taken care of stuff on my end, as instructed.

[Snip]Are there any other posts that people might want rescuing? The WoW lot must have a bunch. They might want to get on that.

Will go through the WoW section. Have a look-see what we would need to salvage.
 

Ronin Storm

Administrator
Staff member
*ahem* Back on topic, I have no idea how easy it would be to do a static export of the forum database as an archive (ie dump everything down to raw html). I'd guess it wouldn't be too hard, but have no idea how vbulletin arranges things!

(As you'd probably want to do this for some of the dota/wow/whatever posts, I'd hope it would be easy to just extend that to the rest of the board)

We can dump the database as a backup, which contains all the post data.

I guess the question really is whether we can make it safe to transport content en-masse. Haven has suggested (elsewhere) that a forced password reset might handle the game, which is quite possibly what'd happen on a migration anyhow. Maybe that'd make a data migration a welcome thing.

In any case, I think it's time for some consolidation and pruning.
 

BiG D

Administrator
Staff member
Fuck you both.
EDIT2: I don't have to explain myself, I did nothing wrong, Razzak grow up and stop picking fights.
REALLY? Who needs to grow up?

If you want details, ask for details. Don't say the lack of details implies some grand conspiracy. It's offensive to the people volunteering their time to fix this. You should probably THANK the admin team and drop it.
 

Razaak

Well-Known Member
REALLY? Who needs to grow up?

If you want details, ask for details. Don't say the lack of details implies some grand conspiracy. It's offensive to the people volunteering their time to fix this. You should probably THANK the admin team and drop it.

Thanks for saying this in better terms than I could have, Big D. Given the nature of Ki!ler-Mk1's outburst, it's probably a good job I didn't post the first draft of my earlier response :D

Ki!ler-Mk1, if you're going to post offensive and unsubstantiated accusations at the administrators of this site, and then insult anyone who disagrees with you (including a moderator) then might I suggest that you are in the wrong place, and not to bother registering if a new forum replaces this one.

Also, my name on here has one Z and three As, if you find six letters so difficult to type, then copy and paste is your friend.
 

Ronin Storm

Administrator
Staff member
[mod]Okay fellahs, back to topic please.[/mod]

Discussion behind the scenes feels like it's leaning as follows:

Xenforo over phpBB, though jury still out on migration of data.

My main concern is that the current user database has already given too much away and that retaining it in any form increases our future exposure. This may be paranoid.

So, the crux: no user accounts, no content either. We have both, or neither (as unlinked user content isn't very useful). If you have a strong opinion, now would be a good time.

Why Xenforo? More focus on better/modern technology (HTML5 & more web 2.0 vs XHTML for IE6 compatibility). Also, potential for direct Steam user account integration (you'd potentially be able to sign-in with your Steam credentials) and decent shoutbox functionality available (though the same is true for phpBB). Both are extensions to the base forum.
 

Ronin Storm

Administrator
Staff member
[mod]I think you've seen the message above now. Seriously, not got time for modding and board crisis management. Let's keep to the topic please.[/mod]
 

Ronin Storm

Administrator
Staff member
For self-enlightenment purposes only, may I ask if the additional Xenforo functionalities are really all that that they're worth the additional expense over just plain Open Source phpBB?

Not fighting with out of date technologies for backwards compatibility we don't need is a fairly big plus. HTML5 FTW.

Also, potential for single sign-on using Steam credentials (for those who want to).
 

Ronin Storm

Administrator
Staff member
So, in the absence of objections (there've been none so far), we'll be switching from vBulletin to Xenforo. All being well, this will happen this weekend, likely Saturday.

The rough plan:

  1. This board will be archived (publicly, for now) but posting will be disabled. The archive board will be available at a different URL, which I'll post at that point. This will probably happen Friday evening late.
  2. The "live" board will be munged and cleaned pending data migration. This will not affect the archive data/board.
  3. The site here at games.thehavennet.org.uk will be flushed and a new Xenforo instance will spring up in its place. In all likelihood, we'll be restricting access to this domain while that is going on to prevent mistakes.
  4. Data, including user accounts, will be migrated to the new board. More on that below. It is also possible this step will fail, in which case we'll be starting from scratch.
  5. Further configuration for the new board will occur and functionality testing. We may grab a person or two to check that things are working from an ordinary user account perspective.
  6. Once we're happy with the initial set up, we'll open the main board to the public.
  7. Sometime after that, we'll close the archive board permanently.

On user accounts and (my) paranoia:

We anticipate that, at the very least, username, email and hashed passwords have been compromised, and I'd anticipate salt too. We intend to reset all user passwords and re-salt them. As a result, all users will need to use the "lost password" functionality on the new board to get access to their user account.

That leaves username and email addresses as vulnerable. Unfortunately, with many of these boards the username is publicly displayed in many places and, with only the barest of inspection, can be easily associated with a user id. One should probably consider one's username as public already. Given that, I'm not going to worry about usernames.

So, just email. In this case, I suggest that if having your email address compromised is worrisome, you should change it on the new site. You will most likely be able to do this yourself via your user control panel, though if not we can change it manually for you.

Given that, I'm now feeling less worried about transporting user data across. Just remains to see if it will actually work at all...
 

Nanor

Well-Known Member
I like the look of Xenforo. Hope the transition goes smoothly and thanks for doing this.
 

Ronin Storm

Administrator
Staff member
Okay, I can confirm that we'll be switching to Xenforo as I've now bought a license and successfully deployed it in a test environment on this server. Had a single problem during initial installation that I'll be looking to prove I've fixed tomorrow evening, but for now I'm comfortable that the basics work and this is a viable solution.

I've not yet experimented with data migration. I don't doubt I'll be doing this whole process two or three times before I'm happy that I'm going for a final run...
 

Kasatka

Active Member
I'm all for a move to new, sleeker forum setup - ive found these forums to feel quite bloated when you only use a small subsection of them. A simple Chat/Games/Admin/Other setup would be sweet, as would any Steam integration as i think the majority of us play games through that nowadays anyway.
 

Ronin Storm

Administrator
Staff member
I'm all for a move to new, sleeker forum setup - ive found these forums to feel quite bloated when you only use a small subsection of them. A simple Chat/Games/Admin/Other setup would be sweet, as would any Steam integration as i think the majority of us play games through that nowadays anyway.

I don't think it's going to quite that streamlined but I'd like to at least cut around 50% of the currently publicly visible forums, or nest them in a more compact way at the very least.

Just testing a full database import into the test instance now based on current vB data. I plan to do some more severe pruning before we actually get to doing this live over the weekend.
 

Ronin Storm

Administrator
Staff member
Nope. At least not as far as I can tell. However, we'll be having something equally cool on the new board. Just wait. :)

Actually, they do come across but as Likes not Cookies. Close enough.

Still, we'll likely go with the cool thing here regardless. ;)
 

Ronin Storm

Administrator
Staff member
Just testing a full database import into the test instance now based on current vB data. I plan to do some more severe pruning before we actually get to doing this live over the weekend.

So, initial tests on importing have gone ahead successfully. A little heads-up on some of the impacts you're likely to see:

Social Groups / Social Forums will largely disappear. There's no built-in compatible functionality for them. There's two or three Social Forums that have received a fair bit of input and their content will survive in one form or another but they may rejoin the main board or, if necessary, be changed into some sort of private forum for specific members.

Titles: we've been using these little userbars for titles for a bit but they'll all break on import. There's an alternative approach that may be possible for them but this may take a bit of time to get up and running.

Passwords: just a reminder that ALL passwords will be scrubbed and you'll need to use the "forgot my password" functionality on the new forum to reset it. This will require that your email address is up to date. If it's not, fix that now.

User profile fields: some profile fields won't translate well or duplicate those in Xenforo but don't merge with them. I'll be purging those fields before migration. Gender is one. Biography is another.

Permissions: vBulletin and Xenforo think about permissions a bit differently. We're going to try to keep it simple but if you've lost access to something you think you should be able to see it either has been deleted, merged with another forum, or we've just not fixed that bit yet.

Forums: we anticipate some heavy purging and merging of existing forums to simplify the migration process and also help this place feel a bit more lively by being a bit (lot?) more compact. You can make your requests for stuff to keep here. Be advised that I'll be migrating based on what makes it relatively easy, so this is one area you may have to lump it if you don't like it. :p

Smilies: expect smilie meanings to change in old posts because Xenforo translates them a bit differently at times. Just be advised.

In-forum links: some (many?) links are going to break between threads. I think it'll happen more in my heavily cross-linked wall-o-text posts. Just know it's going to happen, I guess.

That's all I've got for now. Happy for questions, though if it comes to "making things work" or "answer lots of questions" I'm going to concentrate on the former rather than the later over the next few days while we get this migration out of the way.

Oh, finally:

This instance of THN will be closing down Friday night sometime. You'll see a password request dialog before you can even visit it and you can't have the password because it's there to protect the forum while I decommission and prepare for final migration on Saturday.

An archive will be available at:

http://archive.thehavennet.org.uk

You'll be able to log in and such but functionality is not guaranteed and anything new you do on there will be lost. That URL is temporary and everything in it will be purged once I've verified that we've migrated successfully.

Post migration, we'll be back at this URL just like always.

http://games.thehavennet.org.uk

I reckon you've got around 22 hours of vBulletin left...
 
Top