HTTPS is less (not more!) secure than HTTP
OK, the title is supposed to be provocative, and maybe unbelievable. But just hear me out...
Just what is security? In an information centric world, it means not divulging information to "untrusted" parties.
So, what happens when you communicate with a web server from your handy cell phone (they even call it "Handy" in Germany!)
Your cell phone first has to tell the phone company it wants to talk, and because the phone company is not giving out data (or phone) service for free, the phone has to identify itself as associated with a paying account. In the US, this typically means the phone company knows who the caller is, since most accounts are on a monthly cycle and the phone company wants to keep billing you. Of course, you could be using a prepay account, in which case they may only know which SIM card is making the call. On the TV action shows they call this a "burner" phone. Because the connection to the phone company is over the air, while the phone company does not know precisely where you are, it knows which cell tower your phone is talking to, so it knows approximately (within a couple of miles) where you are. Because the phone connection is over the air, anybody with the right equipment can listen in, or jam the communication.
Once the phone company is satisfied, it sends your message to the phone company or ISP servicing the webserver. But because of the shortage of IPv4 addresses, the message goes out using one of the IP addresses owned by the phone company. Because the phone company needs to send a response back to you, it remembers which address it used for you, although it can change. This feature is called NAT (Network Address Translation). Thus the webserver typically does not know who the sender is, nor can it correlate messages sent by the same person unless that person chooses to expressly identify himself or herself to the webserver. You do that by logging in and after that your browser sends a bit of data with every message that lets the webserver know who you are. There is a lot of detail about how this is done without allowing somebody else to counterfeit your identity, but that is basically what happens.
In between your ISP (or phone company) and the webserver's ISP is the Internet, where anybody along the transmission path is presumed to be able to look at the messages going back and forth and do whatever they want with them, including throwing them away, altering them, duplicating them or inserting their own messages. In reality, given the commercial requirement of good service, most messages make it through unaltered, and the ISPs work hard to make sure that is true. Any path containing slow or malicious nodes is quickly blacklisted and an alternate path is found. Given the business need not to let the ISP at the other end (or especially the ones in the middle) steal their customers, they will do anything they can not to place their customer information on the Internet. It took a law to get them to post caller ID, and another law to incorporate GPS and report location information.
OK, now you are Ms. Average Jane. You have to trust your phone company with your identity, otherwise you are not going to get service. You bought the phone from your phone company (or maybe you are leasing it) so you have to trust them about the workings of the phone, including any encryption it may do for you. The phone company does not care what messages you send or receive unless a government gets involved, in which case they are going to respect the interest of the government rather than yours. The government does not care what messages you send and receive, so while they may be looking at your messages, they wont be doing anything about it. So you can trust them all. Who you can not trust is your neighbor with listening equipment who can embarass you, or steal your identity and thereby your wealth, but that is unlikely because such equipment costs real money and requires significant technical expertise, which they are unlikely to have. The other party who is likely to break security is the webserver itself - especially if you are browsing before a transaction and collecting information before a purchase - they are exceedingly likely to spy on your browsing and alter the terms of the transaction based on what they observe. And this is where HTTPS is less secure than HTTP (for you, Ms. Average Jane), because at this point there is no difference between HTTP and HTTPS in the availability of the content information. But there is a big difference in the ability of the webserver to identify your phone as being the one that did all the browsing prior to the transaction - because HTTPS for efficiency purposes reuses the session/encoding key, the webserver can connect the browsing to the transaction even if you (Ms. Average Jane) did not log in while doing the browsing, and blew away your cookies and DOM storage before embarking on the transaction. It wont matter if you switch ISPs or your location, or your ISP switches your address, or you use TOR, because you are still using the same phone and browser, and session/encoding key.
The original purpose of the HTTPS system was to ensure that the client (you, Ms. Average Jane) could be certain you were talking to the right webserver, have a private conversation with that webserver, and that you could identify yourself to the server only when you chose to do so within that private conversation. This was extremely inconvenient for many webservers conducting real transactions, because a lot of profit rides on knowing your interests before the terms of the transaction are determined. Webservers therefore first responded by requiring you to login and identify yourself before showing you any information, but ran into the wall of average street smart humans who are reluctant to browse and visit sites where they have to identify themselves first. Imagine being asked for your driver's license before you can look at cars in a car dealership, and having a salesman follow you around while you are looking! In an old style market, a boy would be assigned to follow a prospect when she entered a market, and paid to bring her in and report to the store owner her interests while she was bargaining for an item. Google is that boy in the modern world.
OK, now you are Mr. Not So Average Joe, working on some large or important deal. That nosy parker neighbor is now an industrial spy, with significant resources and technical acumen. You appreciate the 2048 bit encryption HTTPS that the webserver of your deal partner is using to prevent leaks. But your phone came preloaded with garbage root certificate authorities who will issue a certificate for any purpose (including modifying the software on your phone) to anybody who pays enough money, and they paid your phone manufacturer to install their root certificate in your phone, when it was manufactured and before they knew you were going to get such a phone. Mr. Industrial Spy identifies Garbage Root as a certificate authority your phone uses regularly, pretends to be your deal partner and pays Garbage Root for a certificate that says his webserver is your deal partner's web server and then uses a fake cell tower site near your home to redirect your communication with your deal partner to his webserver, for a man-in-the-middle attack. You, confident in the 2048 bit encryption, fail to take the precaution of direct face to face communication or land line for frank discussions, physical handoff for data transfers and code speak in other scenarios. Your competitor walks off with your most valuable secrets and you are blithely unaware.
Even if your deal partner is aware of the risks of using low grade certificate authorities and gives you a private "public key" certificate to authenticate their server, you cannot delete Garbage Root from your phone because your bank uses them to authenticate their server certificates. They implement "perfect forward security" - meaning they change their certificate regularly and you cant just store their certificate. When your phone sees Mr. Industrial Spy's certificate claiming to be your deal partner's webserver it uses Garbage Root to validate that certificate, and happily carries on blithely unaware that it already has a valid certificate for your deal partner that is different.
There are other ways of redirecting messages to a fake webserver, I just chose the fake cell tower scenario because that does not require low integrity on the part of anybody except Mr. Industrial Spy.
The possibility of doing this is what the HTTPS system was designed to guard against - to prevent a man-in-the-middle attack. Unfortunately, commercial reality kills the foundation of the HTTPS public key system - "trusted" root authorities are fallible and the trust placed in them can be misplaced.
The reason I say HTTP is more secure for this scenario is twofold. One, the existence of a strong public key system makes people careless even in scenarios where the attacking entities have significant resources to mislead or corrupt, while a simple private key distribution system would reduce the number of players that have to be trusted; and two, the HTTPS system as designed enhances the discoverability of "trusted" resources that can be corrupted.
Finally, pretend you are an entity like Ms Merkel against whom government resources are directed.
I should really pick a more noxious personality for my example, with the restraint level of the attacker as close to zero as possible. But I pick Ms. Merkel because it actually happened to her.
In this scenario, most of the actors in the little communication scenario we started with have been corrupted (or influenced, if you want to state things positively) to participate in the attack. The only two non-participants are the webserver and the end client who are trying to keep their communication secure. My contention is that the HTTPS system as it exists today has been purposely designed to fail in this scenario. Put another way, it has been designed to enhance the security of the existing world order against rebels, by making it as impossible as possible at multiple levels to engage in a completely secure communication.
1) if the trusted root authorities are corruptible, no communication using them can be secure.
2) the webserver picks the root authority for a certificate, so the man-in-middle can choose a corruptible root authority if one exists.
3) the root authorities trusted by a client can be identified by simple observation of his/her communication over a small period of time, allowing corruption efforts to be targeted on these
4) It used to be feasible to conceal the trusted set of root certificate authorities by using offline validated certificates. This is being shut down in many ways:-
a) It is no longer possible to designate a privately distributed certificate as the only certificate for a site
b) With the advent of "perfect forward security" it is no longer feasible to mistrust all public root authorities by preloading the certificates of all commercial websites one wishes to communicate with, as these constantly change.
c) Caching websites like Akamai use their own root authority for the websites they serve with their own servers, making it impossible to rely only on one root authority for a site.
5) "Deep Inspection" technology allows routers to react to more than just the IP address when routing packets. Specifically they can target the public keys used in HTTPS as alternate routing information.
Packets sent using HTTP do not have this issue, although other information can be targeted.
6) It is becoming harder for the client to see what root authorities are being used for a specific website, especially on mobile equipment.
7) EAP increases the trackability of the client, reducing his/her security, and can no longer be turned off by wireless systems.
8)"Public Key Encryption" is possibly a con. The density of usable keys drops off as the width of the key increases suggesting that it might be feasible to store a list of all private/public key pairs. Also arguments from reversible computing engines suggest that it might be possible to build a private key generator for any specific public key encryption algorithm.
9) Communication equipment providers and shipments are tracked. Orders can be intercepted and the hardware can be corrupted before delivery.
10) It is illegal to sell systems where the set of encryption methods supported by the system is extensible by the purchaser.
So,the broad based move to HTTPS going on at this point may not necessarily be enhancing the security of web clients - it may in fact be decreasing their security. Whether this enhances the security of the US Government is debatable.