I can't yet offer you a solution but what I can do is help validate what you are seeing.
Below are some sample ping times for my edison on wifi (802.11n network). Other wireless devices in the house don't exceed ping times of < 10ms (which is isn't great either but not as bad as half a second). I can also confirm that this at least for me, is attributing to nearly unusable ssh performance. So +1, intel gods please provide us with some insight or let us know you guys are ware of this and working on it. I've also seen it affect performance of Zeromq.
64 bytes from 192.168.1.111: icmp_seq=69 ttl=64 time=363.817 ms
64 bytes from 192.168.1.111: icmp_seq=70 ttl=64 time=173.487 ms
64 bytes from 192.168.1.111: icmp_seq=71 ttl=64 time=102.567 ms
64 bytes from 192.168.1.111: icmp_seq=72 ttl=64 time=432.544 ms
64 bytes from 192.168.1.111: icmp_seq=73 ttl=64 time=14.604 ms
64 bytes from 192.168.1.111: icmp_seq=74 ttl=64 time=169.812 ms
64 bytes from 192.168.1.111: icmp_seq=75 ttl=64 time=88.936 ms
64 bytes from 192.168.1.111: icmp_seq=76 ttl=64 time=317.778 ms
64 bytes from 192.168.1.111: icmp_seq=77 ttl=64 time=342.881 ms
64 bytes from 192.168.1.111: icmp_seq=78 ttl=64 time=155.954 ms
64 bytes from 192.168.1.111: icmp_seq=79 ttl=64 time=74.970 ms
64 bytes from 192.168.1.111: icmp_seq=80 ttl=64 time=412.010 ms
64 bytes from 192.168.1.111: icmp_seq=81 ttl=64 time=332.299 ms
64 bytes from 192.168.1.111: icmp_seq=82 ttl=64 time=252.837 ms
64 bytes from 192.168.1.111: icmp_seq=83 ttl=64 time=98.104 ms