Changing Wireshark Link-layer Header settings on Mac OS

This is one of those quick posts aiming to save me and (maybe you) some time the next time I forget this.

On my Mac, I use Wireshark primarily to capture Wi-Fi traffic, in monitor mode. I want to see the Radiotap and 802.11 headers. Usually I leave Wireshark set this way.

On occasion, I actually use Wireshark to inspect higher level traffic – I want to see the IP addresses and TCP/UDP ports etc. I might be troubleshooting an issue and am using my Mac as the client trying to recreate the issue – so I don’t need monitor mode for that. Simple enough – turn it off in the interface settings (Find this button on the Main toolbar Wi-Fi. en0 Wireshark, Today at 9.20.38 PM  to access the menu, then scroll to the right to find the Monitor mode drop down and make sure your Wi-Fi interface has this disabled):

Wireshark · Capture Interfaces Wireshark, Today at 9.30.03 PM

Then just set the Link-layer header back to Ethernet, just like your other interfaces:

Wireshark · Capture Interfaces Wireshark, Today at 9.30.57 PM

Except “Ethernet” isn’t an option. I could’ve sworn that’s what it is set to by default after install…

I can’t believe this still trips me up every few months. I spent half an hour the other day scratching my head, when the trick is simply to restart Wireshark. Close it entirely, reopen it and voila:

Wireshark · Capture Interfaces Wireshark, Today at 9.34.30 PM

Ethernet is back! Also, the 802.11 options have disappeared because we’re no longer in monitor mode. Now I can see Ethernet, IP, and TCP/UDP headers again:

Wi-Fi. en0 Wireshark, Today at 9.35.58 PM

In comparison to capturing 802.11 frames in monitor mode:

Wi-Fi. en0 Wireshark, Today at 9.38.08 PM

I keep forgetting the need to restart Wireshark for the Link-layer options to change #facepalm.

Note: you also need to restart Wireshark after enabling monitor mode before the 802.11 options will show up in the Link-layer header drop down option.

Or maybe it’s just me. I’m confident that I’ll still forget all about this post next time I try to show a University Computer Engineering class how many packets it takes to load the Facebook home page.

It was 781 (including DNS lookups and a couple of retransmitted frames), in case you’re wondering…

PING! Not always what you think! – Meraki Wireless troubleshooting

I’m quite fond of the Meraki dashboard. I’ve seen firsthand how it can enable lean and low-skilled IT departments to manage more of their own networks themselves. The dashboard GUI makes it easy to find status and troubleshoot at a basic level, but it’s still important to actually understand what is going on under the hood.

Here’s an example. If you’ve ever seen the Meraki dashboard, you’ve probably seen the Ping tool on every client status page. Here, a Meraki AP is successfully Ping-ing my MacBook Pro:

pign-success-invalid-ip

Pretty straightforward. Ping the client, client responds, client is online and working, right?

If you have a Meraki Security Appliance, you may stumble across this little note on the Addressing & VLANs page:

ping-is-arp

Wait… Ping is based on ARP? What happened to ICMP?

We may have jumped to conclusions here. As it turns out, Meraki is not using ICMP like most of us would assume. Here’s an example of a few PCAP frames of that same Meraki AP ARP-Ping-ing my MacBook Pro:

directed-arp

Notice this is a directed-ARP; the Meraki AP (MAC 13:da:90) is sending an ARP request to the MBP (MAC 91:75:d8) rather than sending a broadcast. That is, the Meraki AP already knows the MBPs MAC address. But the ARP response tells the Meraki AP that the MBP is alive, and online – just like an ICMP Ping.

This brings about an interesting question. We network engineers often use Ping as a way to confirm that the network is working. A successful Ping means that routing, IP addressing and the physical path are all functioning correctly at layer 3. But if we’re doing a Ping at layer 2 with ARP – would we be wrongly assuming all is well when we get a response, just like with ICMP?

There is definitely some potential to make incorrect assumptions here. In fact, even though that screenshot above of the Meraki AP Ping-ing my MBP has a loss of 0%, at the time, my MBP had an incorrect IP address and was not Ping-able by other devices in the network (well, via ICMP at least). Here’s the PCAP’d ARP frames from the same time as that Ping output:

directed-arp-wrong-ip

Almost identical, except that both the ARP request and response are from 10.11.3.1, when the subnet is actually 10.11.30.0/24. However, the client is still responding, albeit at layer 2, and that’s good enough for the Meraki AP.

Now, I do think this is one of those things where the vendor has made an odd decision to label this as a Ping without being clear about what is actually being done, but it is after all our responsibility as the network engineer to know what we’re looking at. There are similar examples where traceroute can use UDP or ICMP, depending on the OS, and now you know that sometimes Ping is ARP instead of ICMP.

Here’s the relevant documentation:

Meraki Ping Tool

ARGGHH ARRRRRRRP!!!

I run into this issue several times a year with a certain local DSL/Fiber service provider and ASA firewalls. Sometimes it consistently occurs after an outage, others it is seemingly random. This is the relevant debug ARP input (modified for confidentiality):

?arp-req: generating request for 16.197.160.254 at interface Provider1
arp-req: request for 16.197.160.254 still pending
?arp-req: generating request for 16.197.160.254 at interface Provider1
arp-req: request for 16.197.160.254 still pending
?arp-req: generating request for 216.197.160.254 at interface Provider1
arp-req: request for 16.197.160.254 still pending

What makes this a REALLY bad thing is that 16.197.160.254 is the ASA’s default gateway. No internet access for you.

Unfortunately, I have little (no) visibility into the provider settings, but I can hazard a guess that there is some sort of spoofing protection in place. Sometimes a reboot of the ISP modem will fix the problem, but often times we have to call the ISP and, while trying to explain ARP to tier 1 support should NOT be too difficult, it is typically an exercise in frustration. Not surprising, often we are asked to plug a laptop directly into the modem with the ASA’s static IP programmed on its NIC, which works, and causes the ISP to cheer “NOT OUR PROBLEM!” Of course, if I plug a laptop directly into the Provider1 interface on the ASA, it gets an ARP reply right away and communication to the “gateway” also works just fine. Ultimately, I have not been able to find an easy, repeatable fix from the side I have control over (the ASA), but sometimes the ISP clears an ARP table to solve the problem.

Something similar occurs from time to time with this ISP with NAT/PAT IP addresses that are NOT the ASA’s interface address. In this case, we can PCAP traffic leaving the ASA with the appropriate IP and MAC towards the ISP, but the ISP will never forward the return traffic to the ASA – the gateway doesn’t create an ARP entry for that IP.

In this case, an easy fix is to temporarily change the ASA interface IP to match the NAT IP. This causes the ASA to generate a gratuitous ARP and suddenly the return traffic gets delivered.

This is certainly different from the first case, where no amount of restarting or GARP seems to convince the ISP gateway to reply to the ASA ARP request for the gateway’s MAC, but I suspect the cause may be related.

I’m currently waiting for the ISP to determine if an engineer who actually knows how to log into the gateway router and look at an ARP table does indeed exist, or whether I’m more likely to watch a unicorn run a red light on my commute home. In the  meantime, if anyone can explain what this ISP is doing to cause this behaviour, I’d love to hear it.

 

Wireless Professionals Compensation Survey

Today, Wednesday, November 30th there will be a one-day independent survey to gather information on the current state of compensation in the Wireless LAN community. We respectfully request your participation in this 90-second survey. The current results will be available to all survey takers as they complete the survey for instant feedback. Later, the complete results will freely reported back the entire community.
Thank you for your support and participation!
http://surveymonkey.com/r/wlccs2016 – Wireless LAN Community Compensation Benchmark

CWNE #190

cwne-brennan-martinI woke to some exciting news last week. An email from CWNP informing me that my CWNE application had been approved and that I am now CWNE #190!

This is a huge honor, and I am humbled to be part of the CWNE community. While I like to think that I have picked up some skills over the past decade, I owe a great deal of thanks to people whom I have met over the past year  who have mentored and encouraged me to engage in the Wi-Fi community.

Thanks to Keith ParsonsBlake KroneDavid WestcottPeter MackenzieMartin Ericson for sharing your insights and invaluable knowledge!

Onwards…

 

Cisco Live! recap – Tuesday

I was pretty exhausted Tuesday morning. With two young kids at home, I run on a minimum of sleep already, and all of the activity of CLUS pushed me over the edge very quickly. When the alarm went off Tuesday AM, I had to make a command decision – I knew I was not going to be able to keep up the pace set at the start without recharging my batteries, so I reset my alarm and went back to sleep.

Yep, I was a rookie at CLUS, this year, and I’ll try to be better prepared next time. Unfortunately, that means I had to miss my morning session “Fog Computing at the World’s Largest Copper Mine”. Which it too bad, because I spend a lot of time in mining and we have been looking into Fog Computing a lot lately.

The presentation is missing from the link above, but thankfully the session video is there. Funny that the first thing presenter Francisco Soto says is “Thanks for waking up early for this.” Now I feel terrible.

RowellRobMitch, and I had a meeting setup before lunch with Jerry Olla from Ekahau. I was stoked to be meeting some of the Ekahau crew! Jerry sprung for coffee, and that coupled with a little extra beauty rest meant I was having a pretty good start to day #2! Thanks Jerry!

Jerry answered some of our nagging questions about using Ekahau Site Survey. ESS is indispensable in the WLAN design stage for helping to develop real requirements and turn them into actual channel plans, power settings, antenna coverages. Without a tool like this, many people are just “eyeballing” WLAN design. Jerry also showed us a neat little project he had been working on using an ODROID C2 to build a custom iPerf server.

Jerry is awesome. We may have thought that there was some potential to abuse our new insider relationship with Ekahau to get our hands on some sweet, sweet swag:

Buoyed by the warm reception from Jerry, and our unrequited love for Ekahau, we weren’t ready to part ways just yet, so we headed over to WoS to meet the rest of the Ekahau crew!

Jussi Kiviniemi is well known in the Wi-Fi community, and an excellent representative of the Ekahau team. I was excited to meet him in person he is all class! The whole Ekahau crew just has the right approach to being part of the community. For example, Hannele managed to dig out some primo swag in return for some testimonials:

Maybe I was excited most of all for the legendary Finnish candy. I wear my Ekahau Polo to work all the time.

I may have gotten a little carried away:

Also this happened. No explanations will be forthcoming.

Big Kudos to Ekahau for their continued community support and participation. Also a well deserved shout out to Robert Bartz!

There were TONS of other badass vendors to meet!

Check out Smitty and the team at Acceltex. They’ve got a nice spread of antennas and even better some well thought out brackets and mounts for their own gear and the Cisco APs with internal and external antennas as well. Meru Mitch swears by these guys.

We are all pretty pumped this summer about the new Aircheck G2 from NETSCOUT, and meeting their social media manager Kendall was another highlight.Robb was doing his best to see if a Aircheck G2 could find it’s way to his home somehow (Kendall, he ended up paying good money for one – so you won that round!). I think Kendall probably saw more of us at CLUS than anyone would have wanted, but if anyone could handle these hosers, it was her.

That wouldn’t be the last Kendall saw of us…

Ok I did also squeeze in a session Tuesday afternoon. BRKEWN-3014 “Best practices to deploy high-availability in Wireless LAN Architectures with Patrick Croak.

  • Patrick had some great advice regarding minimizing the number of SSIDs:

BRKEWN-3014.pdf (page 17 of 120) Preview, Today at 9.28.03 PM

  • And a good reminder about client based CCI:

Check out the presentation link above for all of the good stuff.

I was looking forward to the evening events on Tuesday, namely… the CCIE Party! Steve, Mitch and I were off to the Hard Rock Hotel for a beach party.

With Food, Music and Beer taken care of, it was back to meeting some familiar names.


Brian McGahan with INE, whose videos are in part responsible for me qualifying to attend this party. Brian was pumped about my tramp stamp.

Amy Renee is a well known ginger CCIE with a well known sparkly bat. My kids are gingers, so we cool.

I also had the pleasure of meeting and shooting the breeze with Fish herself. I’m confident that I learned something just standing near her.

I learned that Tom Hollingsworth is also a fan of the CCIE tramp stamp, and I was glad to meet one of the minds behind Tech Field Day. I’d love to participate in a TFD or better yet Mobility Field Day, so fingers crossed!

The biggest surprise though, would come from closer to home than expected, when I had the pleasure of meeting another fellow Canuck and Wifi guru @wirelessstew!
I was blown away when Stew said that, not only did he know who I was from my (very new) online presence, but that he had hoped to meet me and find out more about me and my mobility experience. Stew would turn out to be the final, and likely most influential of the hosers group of Canadians (Stew, Steve, and myself), and honorary Canadians, Robb and Meru Mitch.

Us Canadians were a bad influence on Mitch:

But the night was still young so we decided to brave the Vegas heat and check out some more of the CLUS festivities. It was only logical that we take our Canadianized Meru Mitch to the Cisco Canada party. Beer was once againMore beer, music and food, although we never did figure this part out:

So now I have to figure out how to introduce Mitch to poutine…

TL;DR: Tuesday was the meet/make friends/network day. CLUS is an amazing opportunity to make connections and I have already benefitted from my new connections.

Also, I was glad I slept in.

Stay tuned, more shenanigans came Wednesday.