The set up:
My business partner who resides in the SF bay area has a cable connection to comcast at his home. Until recently his connectivity to our colocation has been phenominal… that is until his IP changed. Apparently Comcast has different routing for different IP blocks; this proves to be a bit confusing as you would think it would be easier to load balance if all IPs were announcing the same and being routed the same. The MTR below shows the odd difference.
First his old IP which routed nicely through AT&T and had good stability (low packet loss / low jitter):
Packets Pings
Host Loss% Last Avg Best Wrst StDev
1. 209.104.4.225 0.0% 0.7 0.7 0.6 1.0 0.1
2. tus1.Login.COM 0.0% 0.4 0.4 0.3 0.5 0.0
3. phv-edge-03.inet.qwest.net 0.0% 7.6 20.7 7.1 138.5 41.4
4. phn-core-02.inet.qwest.net 0.0% 7.7 7.6 7.3 7.9 0.2
5. los-core-02.inet.qwest.net 0.0% 18.1 18.2 17.9 20.0 0.6
6. lap-brdr-01.inet.qwest.net 0.0% 18.0 22.3 18.0 43.8 9.2
7. 205.171.1.10 0.0% 20.9 20.9 20.6 22.6 0.6
8. tbr1.la2ca.ip.att.net 0.0% 32.7 36.9 32.4 72.7 12.6
9. tbr1.sffca.ip.att.net 0.0% 32.8 32.7 32.3 33.3 0.3
10. gar4.sffca.ip.att.net 0.0% 32.0 32.1 31.9 32.2 0.1
11. te-9-2-ar01.oakland.ca.sfba.comcast.net 0.0% 35.6 35.8 35.6 36.0 0.2
12. te-9-3-ur02.fairfield.ca.sfba.comcast.net 22.2% 36.6 36.6 36.4 36.7 0.1
13. te-9-3-ur01.fairfield.ca.sfba.comcast.net 0.0% 36.6 36.6 36.5 36.9 0.1
14. te-9-3-ur02.napa.ca.sfba.comcast.net 0.0% 37.5 37.6 37.4 38.5 0.3
15. GE-1-1-ur01.napa.ca.sfba.comcast.net 0.0% 37.4 37.3 37.2 37.5 0.1
16. ???
as you can see packets reach him 100% of the time (the packet loss on hop 12 is just a router shedding ICMP packets as it doesn’t show loss on the proceeding hops).
Now on one of his new IPs (this is no longer his IP and neither is the old).
Host Loss% Last Avg Best Wrst StDev
1. 209.104.4.225 0.0% 0.7 0.8 0.5 20.6 1.8
2. tus1.Login.COM 0.0% 0.4 0.6 0.3 23.9 2.2
3. so0-1-0.jr2.phx2.llnw.net 0.0% 13.4 13.6 13.3 33.7 2.1
4. ge3-1.fr4.phx2.llnw.net 0.0% 13.3 13.3 13.3 14.5 0.1
ge3-2.fr4.phx2.llnw.net
5. tge2-1.fr3.phx1.llnw.net 0.0% 13.3 13.7 13.3 21.8 1.5
6. ge-4-2-0.ar5.phx1.gblx.net 0.0% 23.3 26.5 23.2 110.3 13.9
ge-4-2-0.ar4.phx1.gblx.net
ge-3-2-0.ar4.phx1.gblx.net
ge-3-2-0.ar5.phx1.gblx.net
7. ge2-1-10G.ar4.LAX1.gblx.net 0.0% 27.2 35.0 23.2 234.4 39.5
te7-2-0GE.ar4.LAX1.gblx.net
8. 68.86.85.82 31.6% 24.7 24.6 23.9 32.9 1.2
9. Comcast-IP-Services-Sunnyvale.TenGigabitEthernet3-1.ar1.snv2.gblx.net 47.0% 33.1 39.4 33.0 198.0 24.8
10. 68.86.90.166 70.9% 34.7 35.6 34.6 53.2 3.2
11. te-9-2-ar01.oakland.ca.sfba.comcast.net 8.5% 39.3 39.8 39.0 69.8 3.1
12. te-9-3-ur02.fairfield.ca.sfba.comcast.net 14.5% 40.3 40.2 40.1 41.8 0.2
13. te-9-3-ur01.fairfield.ca.sfba.comcast.net 0.9% 40.2 40.2 40.1 41.3 0.1
14. te-9-3-ur02.napa.ca.sfba.comcast.net 2.6% 41.2 41.2 40.9 58.8 1.7
15. GE-1-1-ur01.napa.ca.sfba.comcast.net 0.9% 41.0 41.0 40.8 41.5 0.1
16. ???
You can see the horrible packet loss and 5ms more latency along with what looks like saturation between comcast and global crossing.
Come on Comcast you make billions a year, how about providing a quality service?
-Scott