Netflix and ISP Throttling bypassed by VPN [Solved]

After Reading This Posting, consider reading the posting I have up on the Amazon Free Tier VPN Server I have up.  Getting VPN Services for a year while being able to control both ends of the tunnel is something that’s really nice.  If you sign up using my EC2 Link here below, I’ll give you support for free and that includes setting up the VPN server to your specifications.

Sign up for EC2 link through me.

I also have a post here that you should consider reading as well.

It just absolutely amazes me that my ISP (Comcast) will throttle Netflix feeds to me every 30-40 seconds.  This basically makes HI-Def movies completely unwatchable.  Now keep in mind that I pay a small ransom for a 50 Mbps Down / 10 Mbps Up feed and I should have no reason to be denied Netflix on my device.

After sitting around and thinking about the problem, I’ve concluded I needed to get through the Comcast throttling infrastructure; After all, I want what I paid for and Comcast isn’t supplying it.

I understand that Comcast and Level 3 are in a pissing contest over extra fees to send Netflix data over.  Interestingly enough, if I’m watching from a mobile device, there’s no pausing.  So they’re clearly causing problems with Television connected devices and HD Feeds.  In other words, to watch a 45 minute show might take 2-3 hours.  I don’t know about you, but that’s just not acceptable to me.

Lugging around my ipad or phone to watch a show or movie is also unacceptable.

At first, I thought it was Netflix’s CDN, but after doing some research and just a little common sense, There is no way the CTO and infrastructure gurus at Netflix and their CDN could be that incredibly incompetent.

So I did a little test.  I configured a pptp tunnel on my laptop and did some streaming via my VPS that happens to sit outside of Comcast.  What do you know?  It seemed to work quite well.

No drops, pauses, etc.  Turn off PPTP and wow! It happens again!

So this led me to have to come up with one simple solution.  Reroute my traffic outside of Comcast via VPN and run it that way.

A few weeks ago, I wrote an article about pfsense and how it was a pretty awesome firewall.  What’s great about it is it gives you complete control over how things are routed!  I set up my firewall to establish an OpenVPN connection to my VPN provider and route some traffic via the VPN and some via the regular comcast network (outside of the vpn).

Why would I do that?   Simply, I have voip phones with a PBX in the cloud and it makes no sense for me to add much more latency to the VOIP traffic so I decided to route the traffic from the sip proxy outside the VPN while everything else goes through the VPN.

NOT ALL VPN PROVIDERS ARE THE SAME

1. Many providers block ports.  This can cause a major problem.  It causes things to not work.  For example Bittorrent, other VPNs, Samba / Windows File Sharing, etc.  The list can go on and on.  This is a BIG problem.  Unfortunately, they don’t tell you until they’ve taken your money.  Once that happens, seriously, good luck trying to get it back.  It’s gone for good.

2. I’ve done extensive research by calling and chatting with the VPN providers as well as trying them out.  What did I find out?  Well.. They all tell you what you want to hear and rarely deliver on the promise.

3. Shopping by price is like shopping for a used car.. You get what you pay for.  Notice how all VPN providers state things like, “Unlimited Bandwidth”?  What they don’t tell you is how much connectivity they really have to their VPN servers.  Many VPN servers are over-subscribed.  Remember, a VPN is about encryption.  There are two major things that will cause problems in your case:

• Network Connectivity from the VPN server
• CPU and RAM on the VPN server

For example, let’s say a server has a 100 Mbps ethernet card in it and the server resides in a datacenter with 1 Gbps of Bandwidth, but your VPN provider is subscribing to 1000 Gigabytes of Transfer a month.

Now, let’s say there are 200 users on your server.  Each one pushing an average of.5 Mbps over the course of the month.  This works out to a completely flooded ethernet interface and not enough bandwidth and CPU to handle the connections. What effect does that have on you? A REALLY SLOW VPN.

1000 GB of transfer  a month is roughly equal to: .75 GB per second or roughly 768 Mbps.  Only one problem.  The server has a Gigabit Ethernet connection.  Real world measurements + overhead + encryption + translation (NAT or Bridging) will not give you that speed.  If even 50% of the people decided to watch Netflix that night (a good policy) or any type of streaming media, you’re going to get horrible speed and such.

It’s not uncommon for a VPN provider to oversubscribe their servers.  That’s how they make their money.  They’re betting that not all the users will be on the server at the same time and watching videos or transferring large chunks of data (backup, ftp, video, audio, torrent)… You get the idea.

The other thing to think about is locations.  How many locations and which backbones are those servers attached to?  Realistically, I can get into that, but it would be a very technical discussion and definitely the subject of another post.

Locations: Make sure your VPN provider has multiple locations and servers you can connect to.  Make sure you can establish a VPN to that location and do a SpeedTest to that with the VPN on.  You’ll quickly see the difference.  Do your speedtest from the location at three times:  Prime Time (6-10pm), Morning (8-11am), and lunch (12pm -2pm).  This will give you an idea of how your vpn connection will perform.

Make sure switching locations should be easy and doesn’t require a techsupport call or ticket.  VPN providers are notorious for not providing good or any techsupport.  You’re going to get a short answer most of the time.

So I’ve spent my time and money to evaluate about 10 VPN providers. I can tell you that it was a grueling experience and have come to the conclusion that there was really only one provider out there for me that even seemed remotely feasible.

The provider was StrongVPN.  I’m not naming the other providers, due to the fact that I don’t trash people or want to hurt their businesses.

First, let me show you a diagram of what we’re accomplishing:

Please keep in mind I just threw this together quickly and it proves the point.

1. My LAN sits behind the pfsense router / firewall.  It’s Natted.  All my devices, phones, computers, tablets, etc. are all there.

2. The Pfsense Router has one WAN interface to Comcast.

3. I route my phones / VOIP service through the WAN gateway and not the pfsense gateway to lower excess latency, because VOIP is ultra sensitive.  My pbx sits in a datacenter.

4. A separate OPENVPN tunnel runs to StrongVPN.  My final provider of choice.  They give me a static IP and they’re not on the Comcast network, although the VPN travels through Comcast to get there.  The difference is, it’s encrypted traffic and Comcast can’t see what the traffic is (Netflix, FTP, etc).  If Comcast were to throttle VPN connections, all holy hell would break loose, so they’re not going to do it unless they’re just infinitely idiotic.

5. I route all my other traffic through the OpenVPN tunnel to StrongVPN, because it just makes sense.  If Comcast is throttling, they’re doing deep packet inspection.  They’re literally looking at your traffic.  This is a HUGE breach of privacy so I choose to send everything else through.

6. If comcast throttles VOIP traffic, they’re in violation of FEDERAL FCC LAW so they can’t throttle the traffic. This leaves my VOIP traffic low-latency. Comcast or any other ISP doesn’t want the fines or the class-action lawsuit associated with this.  They leave VOIP alone.

Now to the fun stuff; configuring pfsense to work with StrongVPN.

1. Sign up for StrongVPN and select the standard usage package.  This gives you access to OpenVPN and PPTP in the United States.  If you were to try to access Netflix from Canada or any other country, you’re not going to get any videos, because Netflix probably hasn’t struck a deal with the studios for the other countries.

2. Download their greeting message (or you’ll get it via email).  At this point, you can just follow the directions to implement the package on your pfsense box, but I suggest logging into StrongVPN and selecting an OPENVPN location that is close to you and has the available bandwidth to perform what you need.  StrongVPN gives you a speedtest tool on their webpage and it works quiet well.  So I would just test locations near you and find one that can handle your bandwidth.  Select it, select a port you’re going to use in the StrongVPN control panel and a new package of certs will be sent to you.  These are the ones you’re going to need.  The original ones you got will not work with your location so don’t bother with those.

3. unzip the package and you’ll see the following files.

[singlepic id=39]

 4. Now you’re going to go into your pfsense machine via web and go to system –>Cert Manager. Click on the CAs Tab and hit the “+” button to insert a new CA.  (I’ve included the screenshots in a gallery at the end of the post, but I will also be showing the images throughout this post.

[singlepic id=53]

5.  Open up the ca.crt file in notepad and copy and paste the entire contents into the first box. Give the CA a descriptive Name and save it.  You’ll now see your CA in pfsense as follows:

  [singlepic id=25]

6. Click on the Certificates tab and hit the “+” button and fill in the values as follows: Use the screenshot below as a reference.

  [singlepic id=38]

You’ll have a file called ovpn[some number].crt, open it in notepad and copy and paste the entire contents into the first box.

You’ll also have a file called ovpn[some number].key and you’ll want to do the same into the second box.  (open in notepad and copy and paste the contents of the file into the second box)

Give the certificate a descriptive name and just hit save.  You should now have a certificate under the certificates tab just like you have a CA entry under the CA tab.

7. Now we’re going to configure the openvpn connection.  So go to: VPN–> OPENVPN  and click on the clients tab.  Click on the “+” button and use the following screenshot as a guide.

[singlepic id=29]

In the Server or Host IP address, open the file called ovpn[some number].ovpn and look for the line that starts with: remote [some ip address] [port number]

Enter the values into the form.

Open the file called ta.key and copy and paste the contents into the Enable Authentication of TLS packets box.  Obviously use notepad to open the ta.key file.

Hit save after you’ve double-checked all the settings.

8. Now, you’re going to check your systems logs and under the openvpn tab, the last line you should see is Initialization Sequence Completed.  If you do not see this, it means your settings are incorrect.  Go back and delete everything and start again.

9. Once you see the Completed Initialization message, your tunnel is up to StrongVPN, but it’s far from done at this point.  What we’ve established is an OPENVPN tunnel to StrongVPN at your choice location, but we have not delegated traffic to route through the OPENVPN Tunnel.  Most people would think we would do this with routes, but in pfsense 2.0.x we’re going to do this with Firewall Rules.  It’ll make sense very quickly once you’ve done one.

10. go to Interface –> Assign and hit the “+” sign and you’ll add a new interface (probably called OPT1 with a pull down box next to it.  Select the OpenVPN connection.  It should look like this:

[singlepic id=36]

Obviously, hit the save button.

11. Now Go Back to Interfaces, select the openvpn connection and change the descriptive name to something easy to reference.  I chose StrongVPN.  Hit the Save button and you’ll now see a list of interfaces that look like the picture above.

12. Go to Firewall –> NAT–>Outbound and select Manual Outbound Nat Rule Generation. and hit save.

[singlepic id=27]

 13. Now comes the Magic. Go to Firewall–>Rules and hit the plus sign under the LAN Tab.

Create the following rule:

[singlepic id=31]

Scroll down to the bottom and under gateway, select your openvpn gateway.  Then hit save.

You should see the following:

[singlepic id=43]

Now go to the the WAN Tab and create the same rule.  (Weird,while it doesn’t make sense, if this rule is missing, it didn’t work for me.)

14. Go to the StrongVPN tab (or whatever you named it) and create the following rule by hitting the “+” sign and filling out the form and saving it as follows:

[singlepic id=48]

[singlepic id=49]

15. Go to the OpenVPN tab and create the following rule by hitting the “+” sign and filling out the form and saving it as follows:

[singlepic id=51]

Believe it or not, you’re done at this point.  You should go to http://whatismyip.com to see your current ip and it should be the vpn IP.

If not, just reboot the machine and all should work fine at this point.

If it is working, make a backup of your configuration and save it somewhere safe.

So now comes the question on what to do if we want to route only certain devices through the VPN? (Like a netflix, boxee, xbox360, PS3, etc).  Go to your dhcp manager, assign a static ip for the mac address of the device and create a rule for that ip to route through the gateway of the VPN and all traffic for that device will go through the VPN only.  There is one little caveat though.  These device specific rules need to be before the rules (on top) we’ve just created.  Pfsense interprets rules from top to bottom.  If it matches a rule, it will automatically stop processing the ruleset and execute it.

I really hope this helps considering the pfsense set-up page on the StrongVPN site didn’t exactly work as I had hoped.

If you’re being completely driven mad by this, I can configure this for you (just click on the Hire Me link).  I’ve done this so many times i can do this with my eyes closed.  This little adventure (including VPN provider research) took me over 40 hours and $150.00 of my own money so if you can help donate to the cause, it would be great.  I’d love to recoup my investment.

There are other options as well.  If you happen to have a friend who happens to be sitting in FIOS land or on an ISP that doesn’t throttle, suggest he set up an openvpn gateway for you.  (Also easy to do with pfsense) and you can connect through him/her.  That is more of an unlikely condition as most people value their bandwidth and have absolutely no desire to do this for people considering it could potentially eat all their bandwidth.

**BIG NOTE** DO NOT USE YOUR WORK to run a VMWARE or virtual machine to do this without explicit WRITTEN permission from your boss or it will surely get you fired and possibly prosecuted for misuse of company resources.  NOT A GOOD THING**

UPDATE: It’s been a week or so and I’d like to report it works perfectly.  While there have been a few drops (OpenVPN connection drops), recovery is within 5 seconds.  I’ve had 3 drops within the week.  Each time, I’m back up and running in a few seconds.  Definitely watchable!!!

So until your ISP decides that they want to ban / throttle VPN connections, this is a great solution. Of course, Encrypting Data via OpenVPN can be used for a multitude of things.  OpenSSH, Secure File transfers of Medical / Government / Sensitive Corporate Data, etc.

I’d love to see them explain that one.

After Reading This Posting, consider reading the posting I have up on the Amazon Free Tier VPN Server I have up.  Getting VPN Services for a year while being able to control both ends of the tunnel is something that’s really nice.  If you sign up using my EC2 Link here below, I’ll give you support for free and that includes setting up the VPN server to your specifications.

Sign up for EC2 link through me.

I also have a post here that you should consider reading as well.