JNUC 2015 technical issues

[JayDuff]

Is anyone else having trouble connecting to free wifi at the Guthrie?

My colleague and I each have 3 devices and none of them will connect.

Is the wifi infrastructure here really so insufficient, or are we both doing something wrong?

[rob_brynelsen · Accepted solution]

Hey everyone-- Rob from JAMF’s IT team here. We’re working with the Guthrie staff to see if anything can be tweaked during the event. They’ve put in a huge effort to improve the quality of service this year, but we’ve also brought in more people and devices than ever before.

Here’s what we know:

They have approximately 45 APs total. If we assume 2000 devices, perfectly distributed throughout the venue, that’s an average of ~45 per AP. In reality we’re probably all on the same 15-20 APs in the main performance and common spaces, meaning an average of 100-130 clients per AP. The Thrust Theater had 1030 devices connected yesterday (600+ on 2.4 Ghz). At those numbers, management frames alone (which happen at a slower data rate) can consume the available airtime.

While the individual subnets are /22s, the Guthrie’s IT team reports that they have multiple subnets in a superscope configuration (to facilitate roaming via a single L2 vlan) and that IP address shortages should not be an issue, assuming you can associate.

The venue has a 200 Mbps (symmetrical) connection, and we’re getting 170 Mbps of it. We’ve been saturating our allocation throughout the event, but they have QoS policies and bandwidth management/shaping (not sure on the flavor) to help with this. Even though the per-client allocation would be under 200 Kbps, I was able to pull 5-10 Mbps down and push 20-30 Mbps up yesterday (once associated).

The best thing we can do right now is try and reduce the number of devices competing for the RF. Turn off hotspots, unneeded devices, etc. Echoing georgecm12’s comment-- tether with a cable (and turn wifi completely off) if you can. Also, pause/disable CrashPlan, Dropbox, Box Sync, Google Drive, etc during the day.

Totally love the ideas on here (the XR-6000 looks pretty badass). Feel free to stop me in the hallway (or email me @jamfsoftware.com) with any questions, feedback, tech data, or additional ideas for the venue IT staff. You guys rock.

[DJC]

I think it's just overloaded. I could connect but couldn't load anything.

[__AHN_IT_Admin]

Same deal here. No connection.

[Chris_Hafner]

It's a pretty common issue (well it was last year) given the load we're all putting on it. That said, knock on wood. Things are going OK for me at the moment.

[Aziz]

I live in MN and can chime in.

The Guthrie theater is a performing arts theater. Simply put, the network infrastructure isn't good enough. Right after the JAMF Keynote, everyone tried connecting and the access point (I'm guessing just 1-3 AP's in the main stage room) ran out of IP's to give.

Events like this (tech) don't usually happen here. They're usually plays where they tell you to put your devices away.

[donmontalvo]

LTE disabled works for my iPhone, this was at the Keynote (which was a full house BTW).

WiFi worked after a few tries, maybe DHCP pool runs out? Cringing at the thought of putting my laptop away for lunch, and having to fish for an IP again. :)

[davidacland]

Seems to work on and off for me. It can't be easy providing wifi for an event like JNUC.

[jarednichols]

Wi-Fi is hard. It's why we stand up our own network at our events, even internal-only ones.

[mgrady]

I brought a company Verizon Mi-Fi with me after my experience at the 2013 JNUC. While my reception isn't the best (1-2 bars average) I can manage to VPN, email, and do general work.

[emily]

Yeah I'm basically not closing my laptop lid out of fear of losing an IP at this point.

[georgecm12]

Unfortunately, as a result of the overloaded Guthrie wi-fi, it looks like everyone is tethering, making the wifi spectrum very, very crowded, particularly within the auditoriums.

If I might make a suggestion, as a courtesy, if you are using tethering off of your mobile, consider wired tethering and turning off wifi on your mobile if possible.

[roiegat]

It was the same thing last year. But think about it this way. You have way too many geeks in one building for any Wifi to handle. Think of the thousands of devices in that building...you'd have to design a really super sophisticated wifi system to handle that kind of load. I doubtt the Guthrie has that kind of budget.

[Chris_Hafner]

Honestly... A rented Xirrus Rapid deployment kit should be able to handle it easily. 12 to 24 radio array suspended from the ceiling and a decent DHCP server can crank. They are only a couple $k and can be moved around easily.

[mm2270]

Last year I gave up on trying to get any lengthy established Wi-fi connection after the first couple of hours. I can only imagine the issue is more pronounced this year with the growth in attendance.
The only place it seemed a little better was in the upper floor classrooms between sessions, but that's not a great help when you're in any of the main sessions on lower floors.

[roiegat]

@Chris_Hafner Cool technology. But looking at the documentation it would only be able to handle "up to hundreds of connection" which isn't bad...mean you'd need a couple of them at least.

But it's also the backend that's the issue. Even if they had a very fast connection, your still dividing it up way to much. So you'd have to have some sort of software limiter to keeps things under control.

I personally ended up using my AT&T hotspot most of the time.

[Chris_Hafner]

@roiegat fair enough. I'm just a huge fan of hose things at conferences. Microsoft drags them around everywhere. That said, I'd like to know what the actual setup here is. Besides, this event is only going to keep getting bigger right?

[JayDuff]

I've emailed my Ruckus friends. They specialize in this kind of thing. Take whatever the ISP bandwidth is, divide by 1,000, and that's what you give each client.

They also do temporary mesh networks.... Next year?

[Chris_Hafner]

Just because I had a second. If they wanted to go big a single the XR-6000 is rated for 1792 simultaneous users and connect to a 10 gigE pipe while handling application/traffic shaping on it's own. It's like ~10 grand for the single array but...

[jdavidson]

@donmontalvo You're exactly right. Looks like the Guthrie is running a /22 subnet on their wifi. Meaning they've got a DHCP pool of only 1022 addresses, minus any static reservations, router addresses, etc. 1700 attendees, multiple devices per person, and lets assume the DHCP lease is set to 1 hour. We're all just fighting to get an address -- but once you get one, connectivity is fine. Now we just need to find the right people to beg to bump the subnet up to a 255.255.240.0.

[Chris_Hafner]

You are all brilliant! Now, like you said @jdavidson time to find the proper folks!

[infosecnoob]

Lets just say if "Bob" happened to notice the pool was exhausted today, and then "Bob" fired up wireshark. He may have found the router address. Bob could then manually give himself an already assigned ip and guess the subnet. This will force traffic from "Bob" to the router, switch back to DHCP and may or may not get traffic priority for DHCP requests.

[mpermann]

JAMF needs to get a networking company as a sponsor for JNUC. Maybe they can bring some wireless gear to the Guthrie so there is reliable wireless during the conference.

[davidacland]

That's definitely the plan for macad.uk in London. Get a good wireless provider as a sponsor so we can all have solid Internet.

[emily]

Ciscoooooooooo

Make them prove their love of Apple.

[rob_brynelsen · Accepted solution]

Hey everyone-- Rob from JAMF’s IT team here. We’re working with the Guthrie staff to see if anything can be tweaked during the event. They’ve put in a huge effort to improve the quality of service this year, but we’ve also brought in more people and devices than ever before.

Here’s what we know:

They have approximately 45 APs total. If we assume 2000 devices, perfectly distributed throughout the venue, that’s an average of ~45 per AP. In reality we’re probably all on the same 15-20 APs in the main performance and common spaces, meaning an average of 100-130 clients per AP. The Thrust Theater had 1030 devices connected yesterday (600+ on 2.4 Ghz). At those numbers, management frames alone (which happen at a slower data rate) can consume the available airtime.

While the individual subnets are /22s, the Guthrie’s IT team reports that they have multiple subnets in a superscope configuration (to facilitate roaming via a single L2 vlan) and that IP address shortages should not be an issue, assuming you can associate.

The venue has a 200 Mbps (symmetrical) connection, and we’re getting 170 Mbps of it. We’ve been saturating our allocation throughout the event, but they have QoS policies and bandwidth management/shaping (not sure on the flavor) to help with this. Even though the per-client allocation would be under 200 Kbps, I was able to pull 5-10 Mbps down and push 20-30 Mbps up yesterday (once associated).

The best thing we can do right now is try and reduce the number of devices competing for the RF. Turn off hotspots, unneeded devices, etc. Echoing georgecm12’s comment-- tether with a cable (and turn wifi completely off) if you can. Also, pause/disable CrashPlan, Dropbox, Box Sync, Google Drive, etc during the day.

Totally love the ideas on here (the XR-6000 looks pretty badass). Feel free to stop me in the hallway (or email me @jamfsoftware.com) with any questions, feedback, tech data, or additional ideas for the venue IT staff. You guys rock.

[jarednichols]

Get off of 2.4Ghz. All Apple devices in the last few years support 5Ghz. Leave 2.4Ghz for the massive amount of Bluetooth that I'm sure is going on.

[Chris_Hafner]

@rob.brynelsen Great response, thanks for that! Yet another example of why I love JAMF and JAMFNation so much. A bunch of good ideas all around! So next year 2.5k devices ;-)

[donmontalvo]

Wi-Fi Explorer shows seven 5GHz nodes. No problem connecting in fully packed main stage.

Right-click to show in new window, to get closer view.

[tschaps · Accepted solution]

Regardless of what the Guthrie folks say, @rob.brynelsen, they did not have enough available IP's yesterday with the /22 subnet, supernetted or not. However, overnight it seems they changed their subnet to a /21, doubling the number of available IP's. Much better.
And, better late than never, though I (and probably others) pointed out the insufficient available IP's at last year's JNUC:
https://jamfnation.jamfsoftware.com/discussion.html?id=12318#responseChild71429

It's a herculean effort to satisfy so many geeks, thanks for improving! :)

[DLarson]

Sorry I missed this years JNUC. Last year was tough getting an IP for sure. Even with the change to a /21, the 200Mbps connection seems archaic (of course I'm not paying their bill) and don't forget the OP said, "My colleague and I each have 3 devices..." That's 6 IPs for 2 attendees. I'm not so sure that Rob's estimate is high enough. I hope to see you and lots of IPs next year!