From 00158363b47d916d159d2f3a0e1152783fd66a9c Mon Sep 17 00:00:00 2001 From: Scott Main Date: Thu, 17 Jan 2013 16:58:50 -0800 Subject: [PATCH] docs: add bdc's security doc as training article Change-Id: Iaee006f5e93356cc7cda8e5bff7fc03642329e46 --- docs/html/training/articles/security-ssl.jd | 539 ++++++++++++++++++++ docs/html/training/training_toc.cs | 7 + 2 files changed, 546 insertions(+) create mode 100644 docs/html/training/articles/security-ssl.jd diff --git a/docs/html/training/articles/security-ssl.jd b/docs/html/training/articles/security-ssl.jd new file mode 100644 index 0000000000000..9a6320b8dd39e --- /dev/null +++ b/docs/html/training/articles/security-ssl.jd @@ -0,0 +1,539 @@ +page.title=Security with HTTPS and SSL +page.article=true +@jd:body + +
+
+ + + +

The Secure Sockets Layer (SSL)—now technically known as Transport Layer Security +(TLS)—is a +common building block for encrypted communications between clients and servers. It's possible that +an application might use SSL incorrectly such that malicious entities may +be able to intercept an app's data over the network. To help you ensure that this does not happen +to your app, this article highlights the common pitfalls when using secure network protocols and addresses some larger concerns about using Public-Key Infrastructure (PKI). + + +

Concepts

+ +

In a typical SSL usage scenario, a server is configured with a certificate containing a +public key as well as a matching private key. As part of the handshake between an SSL client +and server, the server proves it has the private key by signing its certificate with public-key cryptography.

+ +

However, anyone can generate their own certificate and private key, so a simple handshake +doesn't prove anything about the server other than that the server knows the private key that +matches the public key of the certificate. One way to solve this problem is to have the client +have a set of one or more certificates it trusts. If the certificate is not in the set, the +server is not to be trusted.

+ +

There are several downsides to this simple approach. Servers should be able to +upgrade to stronger keys over time ("key rotation"), which replaces the public key in the +certificate with a new one. Unfortunately, now the client app has to be updated due to what +is essentially a server configuration change. This is especially problematic if the server +is not under the app developer's control, for example if it is a third party web service. This +approach also has issues if the app has to talk to arbitrary servers such as a web browser or +email app.

+ +

In order to address these downsides, servers are typically configured with certificates +from well known issuers called Certificate Authorities (CAs). +The host platform generally contains a list of well known CAs that it trusts. +As of Android 4.2 (Jelly Bean), Android currently contains over 100 CAs that are updated +in each release. Similar to a server, a CA has a certificate and a private key. When issuing +a certificate for a server, the CA signs +the server certificate using its private key. The +client can then verify that the server has a certificate issued by a CA known to the platform.

+ +

However, while solving some problems, using CAs introduces another. Because the CA issues +certificates for many servers, you still need some way to make sure you are talking to the +server you want. To address this, the certificate issued by the CA identifies the server +either with a specific name such as gmail.com or a wildcarded set of +hosts such as *.google.com.

+ +

The following example will make these concepts a little more concrete. In the snippet below +from a command line, the {@code openssl} +tool's {@code s_client} command looks at Wikipedia's server certificate information. It +specifies port 443 because that is the default for HTTPS. The command sends +the output of {@code openssl s_client} to {@code openssl x509}, which formats information +about certificates according to the X.509 standard. Specifically, +the command asks for the subject, which contains the server name information, +and the issuer, which identifies the CA.

+ +
+$ openssl s_client -connect wikipedia.org:443 | openssl x509 -noout -subject -issuer
+subject= /serialNumber=sOrr2rKpMVP70Z6E9BT5reY008SJEdYv/C=US/O=*.wikipedia.org/OU=GT03314600/OU=See www.rapidssl.com/resources/cps (c)11/OU=Domain Control Validated - RapidSSL(R)/CN=*.wikipedia.org
+issuer= /C=US/O=GeoTrust, Inc./CN=RapidSSL CA
+
+ +

You can see that the certificate was issued for servers matching *.wikipedia.org by +the RapidSSL CA.

+ + + +

An HTTPS Example

+ +

Assuming you have a web server with a +certificate issued by a well known CA, you can make a secure request with code as +simple this:

+ +
+URL url = new URL("https://wikipedia.org");
+URLConnection urlConnection = url.openConnection();
+InputStream in = urlConnection.getInputStream();
+copyInputStreamToOutputStream(in, System.out);
+
+ +

Yes, it really can be that simple. If you want to tailor the HTTP request, you can cast to +an {@link java.net.HttpURLConnection}. The Android documentation for +{@link java.net.HttpURLConnection} has further examples about how to deal with request +and response headers, posting content, managing cookies, using proxies, caching responses, +and so on. But in terms of the details for verifying certificates and hostnames, the Android +framework takes care of it for you through these APIs. +This is where you want to be if at all possible. That said, below are some other considerations.

+ + + +

Common Problems Verifying Server Certificates

+ +

Suppose instead of receiving the content from {@link java.net.URLConnection#getInputStream +getInputStream()}, it throws an exception:

+ +
+javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
+        at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:374)
+        at libcore.net.http.HttpConnection.setupSecureSocket(HttpConnection.java:209)
+        at libcore.net.http.HttpsURLConnectionImpl$HttpsEngine.makeSslConnection(HttpsURLConnectionImpl.java:478)
+        at libcore.net.http.HttpsURLConnectionImpl$HttpsEngine.connect(HttpsURLConnectionImpl.java:433)
+        at libcore.net.http.HttpEngine.sendSocketRequest(HttpEngine.java:290)
+        at libcore.net.http.HttpEngine.sendRequest(HttpEngine.java:240)
+        at libcore.net.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:282)
+        at libcore.net.http.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:177)
+        at libcore.net.http.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:271)
+
+ +

This can happen for several reasons, including: +

    +
  1. The CA that issued the server certificate was unknown
    2. +
  2. The server certificate wasn't signed by a CA, but was self signed
    3. +
  3. The server configuration is missing an intermediate CA
    4. +
+ +

The following sections discuss how to address these problems while keeping your +connection to the server secure. + + + +

Unknown certificate authority

+ +

In this case, the {@link javax.net.ssl.SSLHandshakeException} occurs +because you have a CA that isn't trusted by the system. It could be because +you have a certificate from a new CA that isn't yet trusted by Android or your app is +running on an older version without the CA. More often a CA is unknown because it isn't a +public CA, but a private one issued by an organization such as a government, corporation, +or education institution for their own use.

+ +

Fortunately, you can teach {@link javax.net.ssl.HttpsURLConnection} +to trust a specific set of CAs. The procedure +can be a little convoluted, so below is an example that takes a specific CA from +an {@link java.io.InputStream}, uses it to create a {@link java.security.KeyStore}, +which is then used to create and initialize a +{@link javax.net.ssl.TrustManager}. A {@link javax.net.ssl.TrustManager} is what the system +uses to validate certificates from the server +and—by creating one from a {@link java.security.KeyStore} with one or more CAs—those +will be the only CAs trusted by that {@link javax.net.ssl.TrustManager}.

+ +

Given the new {@link javax.net.ssl.TrustManager}, +the example initializes a new {@link javax.net.ssl.SSLContext} which provides +an {@link javax.net.ssl.SSLSocketFactory} you can use to override the default +{@link javax.net.ssl.SSLSocketFactory} from +{@link javax.net.ssl.HttpsURLConnection}. This way the +connection will use your CAs for certificate validation.

+ +

Here is the example in +full using an organizational CA from the University of Washington:

+ +
+// Load CAs from an InputStream
+// (could be from a resource or ByteArrayInputStream or ...)
+CertificateFactory cf = CertificateFactory.getInstance("X.509");
+// From https://www.washington.edu/itconnect/security/ca/load-der.crt
+InputStream caInput = new BufferedInputStream(new FileInputStream("load-der.crt"));
+Certificate ca;
+try {
+    ca = cf.generateCertificate(caInput);
+    System.out.println("ca=" + ((X509Certificate) ca).getSubjectDN());
+} finally {
+    caInput.close();
+}
+
+// Create a KeyStore containing our trusted CAs
+String keyStoreType = KeyStore.getDefaultType();
+KeyStore keyStore = KeyStore.getInstance(keyStoreType);
+keyStore.load(null, null);
+keyStore.setCertificateEntry("ca", ca);
+
+// Create a TrustManager that trusts the CAs in our KeyStore
+String tmfAlgorithm = TrustManagerFactory.getDefaultAlgorithm();
+TrustManagerFactory tmf = TrustManagerFactory.getInstance(tmfAlgorithm);
+tmf.init(keyStore);
+
+// Create an SSLContext that uses our TrustManager
+SSLContext context = SSLContext.getInstance("TLS");
+context.init(null, tmf.getTrustManagers(), null);
+
+// Tell the URLConnection to use a SocketFactory from our SSLContext
+URL url = new URL("https://certs.cac.washington.edu/CAtest/");
+HttpsURLConnection urlConnection =
+    (HttpsURLConnection)url.openConnection();
+urlConnection.setSSLSocketFactory(context.getSocketFactory());
+InputStream in = urlConnection.getInputStream();
+copyInputStreamToOutputStream(in, System.out);
+
+ +

With a custom {@link javax.net.ssl.TrustManager} that knows about your CAs, +the system is able to validate +that your server certificate come from a trusted issuer.

+ +

Caution: +Many web sites describe a poor alternative solution which is to install a +{@link javax.net.ssl.TrustManager} that does nothing. If you do this you might as well not +be encrypting your communication, because anyone can attack your users at a public Wi-Fi hotspot +by using DNS tricks to send your users' +traffic through a proxy of their own that pretends to be your server. The attacker can then +record passwords and other personal data. This works because the attacker can generate a +certificate and—without a {@link javax.net.ssl.TrustManager} that actually +validates that the certificate comes from a trusted +source—your app could be talking to anyone. So don't do this, not even temporarily. You can +always make your app trust the issuer of the server's certificate, so just do it.

+ + + +

Self-signed server certificate

+ +

The second case of {@link javax.net.ssl.SSLHandshakeException} is +due to a self-signed certificate, which means the server is behaving as its own CA. +This is similar to an unknown certificate authority, so you can use the +same approach from the previous section.

+ +

You can create yout own {@link javax.net.ssl.TrustManager}, +this time trusting the server certificate directly. This has all of the +downsides discussed earlier of tying your app directly to a certificate, but can be done +securely. However, you should be careful to make sure your self-signed certificate has a +reasonably strong key. As of 2012, a 2048-bit RSA signature with an exponent of 65537 expiring +yearly is acceptable. When rotating keys, you should check for recommendations from an +authority (such as NIST) about what is acceptable.

+ + + +

Missing intermediate certificate authority

+ +

The third case of {@link javax.net.ssl.SSLHandshakeException} +occurs due to a missing intermediate CA. Most public +CAs don't sign server certificates directly. Instead, they use their main CA certificate, +referred to as the root CA, to sign intermediate CAs. They do this so the root CA can be stored +offline to reduce risk of compromise. However, operating systems like Android typically +trust only root CAs directly, which leaves a short gap of trust between the server +certificate—signed by the intermediate CA—and the certificate verifier, +which knows the root CA. To solve +this, the server doesn't send the client only it's certificate during the SSL handshake, but +a chain of certificates from the server CA through any intermediates necessary to reach a +trusted root CA.

+ +

To see what this looks like in practice, here's the mail.google.com certificate +chain as viewed by the {@code openssl} +{@code s_client} command:

+ +
+$ openssl s_client -connect mail.google.com:443
+---
+Certificate chain
+ 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=mail.google.com
+   i:/C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA
+ 1 s:/C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA
+   i:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
+---
+
+ + +

This shows that the server sends a certificate for mail.google.com +issued by the Thawte SGC CA, which is an intermediate CA, and a second certificate +for the Thawte SGC CA issued by a Verisign CA, which is the primary CA that's +trusted by Android.

+ +

However, it is not uncommon to configure a server to not include the necessary +intermediate CA. For example, here is a server that can cause an error in Android browsers and +exceptions in Android apps:

+ +
+$ openssl s_client -connect egov.uscis.gov:443
+---
+Certificate chain
+ 0 s:/C=US/ST=District Of Columbia/L=Washington/O=U.S. Department of Homeland Security/OU=United States Citizenship and Immigration Services/OU=Terms of use at www.verisign.com/rpa (c)05/CN=egov.uscis.gov
+   i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 International Server CA - G3
+---
+
+ +

What is interesting to note here is that visiting this server in most desktop browsers +does not cause an error like a completely unknown CA or self-signed server certificate would +cause. This is because most desktop browsers cache trusted intermediate CAs over time. Once +a browser has visited and learned about an intermediate CA from one site, it won't +need to have the intermediate CA included in the certificate chain the next time.

+ +

Some sites do this intentionally for secondary web servers used to serve resources. For +example, they might have their main HTML page served by a server with a full certificate +chain, but have servers for resources such as images, CSS, or JavaScript not include the +CA, presumably to save bandwidth. Unfortunately, sometimes these servers might be providing +a web service you are trying to call from your Android app, which is not as forgiving.

+ +

There are two approaches to solve this issue:

+ + + +

Common Problems with Hostname Verification

+ +

As mentioned at the beginning of this article, +there are two key parts to verifying an SSL connection. The first +is to verify the certificate is from a trusted source, which was the focus of the previous +section. The focus of this section is the second part: making sure the server you are +talking to presents the right certificate. When it doesn't, you'll typically see an error +like this:

+ +
+java.io.IOException: Hostname 'example.com' was not verified
+        at libcore.net.http.HttpConnection.verifySecureSocketHostname(HttpConnection.java:223)
+        at libcore.net.http.HttpsURLConnectionImpl$HttpsEngine.connect(HttpsURLConnectionImpl.java:446)
+        at libcore.net.http.HttpEngine.sendSocketRequest(HttpEngine.java:290)
+        at libcore.net.http.HttpEngine.sendRequest(HttpEngine.java:240)
+        at libcore.net.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:282)
+        at libcore.net.http.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:177)
+        at libcore.net.http.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:271)
+
+ + +

One reason this can happen is due to a server configuration error. The server is +configured with a certificate that does not have a subject or subject alternative name fields +that match the server you are trying to reach. It is possible to have one certificate be used +with many different servers. For example, looking at the google.com certificate with +{@code openssl} {@code +s_client -connect google.com:443 | openssl x509 -text} you can see that a subject +that supports *.google.com but also subject alternative names for *.youtube.com, +*.android.com, and others. The error occurs only when the server name you +are connecting to isn't listed by the certificate as acceptable.

+ +

Unfortunately this can happen for another reason as well: virtual hosting. When sharing a +server for more than one hostname with HTTP, the web server can tell from the HTTP/1.1 request +which target hostname the client is looking for. Unfortunately this is complicated with +HTTPS, because the server has to know which certificate to return before it sees the HTTP +request. To address this problem, newer versions of SSL, specifically TLSv.1.0 and later, +support Server Name Indication +(SNI), which allows the SSL client to specify the intended +hostname to the server so the proper certificate can be returned.

+ +

Fortunately, {@link javax.net.ssl.HttpsURLConnection} supports +SNI since Android 2.3. Unfortunately, Apache +HTTP Client does not, which is one of the many reasons we discourage its use. One workaround +if you need to support Android 2.2 (and older) or Apache HTTP Client is to set up an alternative +virtual host on a unique port so that it's unambiguous which server certificate to return.

+ +

The more drastic alternative is to replace {@link javax.net.ssl.HostnameVerifier} +with one that uses not the +hostname of your virtual host, but the one returned by the server by default.

+ +

Caution: Replacing {@link javax.net.ssl.HostnameVerifier} +can be very dangerous if the other virtual host is +not under your control, because a man-in-the-middle attack could direct traffic to another +server without your knowledge.

+ +

If you are still sure you want to override hostname verification, here is an example +that replaces the verifier for a single {@link java.net.URLConnection} +with one that still verifies that the hostname is at least on expected by the app:

+ +
+// Create an HostnameVerifier that hardwires the expected hostname.
+// Note that is different than the URL's hostname:
+// example.com versus example.org
+HostnameVerifier hostnameVerifier = new HostnameVerifier() {
+    @Override
+    public boolean verify(String hostname, SSLSession session) {
+        HostnameVerifier hv =
+            HttpsURLConnection.getDefaultHostnameVerifier();
+        return hv.verify("example.com", session);
+    }
+};
+
+// Tell the URLConnection to use our HostnameVerifier
+URL url = new URL("https://example.org/");
+HttpsURLConnection urlConnection =
+    (HttpsURLConnection)url.openConnection();
+urlConnection.setHostnameVerifier(hostnameVerifier);
+InputStream in = urlConnection.getInputStream();
+copyInputStreamToOutputStream(in, System.out);
+
+ +

But remember, if you find yourself replacing hostname verification, especially +due to virtual hosting, it's still very dangerous if the other virtual host is +not under your control and you should find an alternative hosting arrangement +that avoids this issue.

+ + + + +

Warnings About Using SSLSocket Directly

+ +

So far, the examples have focused on HTTPS using {@link javax.net.ssl.HttpsURLConnection}. +Sometimes apps need to use SSL separate from HTTP. For example, an email app might use SSL variants +of SMTP, POP3, or IMAP. In those cases, the app would want to use {@link javax.net.ssl.SSLSocket} +directly, much the same way that {@link javax.net.ssl.HttpsURLConnection} does internally.

+ +

The techniques described so +far to deal with certificate verification issues also apply to {@link javax.net.ssl.SSLSocket}. +In fact, when using a custom {@link javax.net.ssl.TrustManager}, what is passed to +{@link javax.net.ssl.HttpsURLConnection} is an {@link javax.net.ssl.SSLSocketFactory}. +So if you need to use a custom {@link javax.net.ssl.TrustManager} with an +{@link javax.net.ssl.SSLSocket}, follow +the same steps and use that {@link javax.net.ssl.SSLSocketFactory} to create your +{@link javax.net.ssl.SSLSocket}.

+ +

Caution: +{@link javax.net.ssl.SSLSocket} does not perform hostname verification. It is +up the your app to do its own hostname verification, preferably by calling {@link +javax.net.ssl.HttpsURLConnection#getDefaultHostnameVerifier()} with the expected hostname. Further +beware that {@link javax.net.ssl.HostnameVerifier#verify HostnameVerifier.verify()} +doesn't throw an exception on error but instead returns a boolean result that you must +explicitly check.

+ +

Here is an example showing how you can do this. It shows that when connecting to +gmail.com port 443 without SNI support, you'll receive a certificate for +mail.google.com. This is expected in this case, so check to make sure that +the certificate is indeed for mail.google.com:

+ +
+// Open SSLSocket directly to gmail.com
+SocketFactory sf = SSLSocketFactory.getDefault();
+SSLSocket socket = (SSLSocket) sf.createSocket("gmail.com", 443);
+HostnameVerifier hv = HttpsURLConnection.getDefaultHostnameVerifier();
+SSLSession s = socket.getSession();
+
+// Verify that the certicate hostname is for mail.google.com
+// This is due to lack of SNI support in the current SSLSocket.
+if (!hv.verify("mail.google.com", s)) {
+    throw new SSLHandshakeException("Expected mail.google.com, "
+                                    "found " + s.getPeerPrincipal());
+}
+
+// At this point SSLSocket performed certificate verificaiton and
+// we have performed hostname verification, so it is safe to proceed.
+
+// ... use socket ...
+socket.close();
+
+ + + +

Blacklisting

+ +

SSL relies heavily on CAs to issue certificates to only the properly verified owners +of servers and domains. In rare cases, CAs are either tricked or, in the case of Comodo or DigiNotar, breached, +resulting in the certificates for a hostname to be issued to +someone other than the owner of the server or domain.

+ +

In order to mitigate this risk, Android has the ability to blacklist certain certificates or even +whole CAs. While this list was historically built into the operating system, starting in +Android 4.2 this list can be remotely updated to deal with future compromises.

+ + + +

Pinning

+ +

An app can further protect itself from fraudulently issued certificates by a +technique known as pinning. This is basically using the example provided in the unknown CA case +above to restrict an app's trusted CAs to a small set known to be used by the app's servers. This +prevents the compromise of one of the other 100+ CAs in the system from resulting in a breach of +the apps secure channel.

+ + + +

Client Certificates

+ +

This article has focused on the user of SSL to secure communications with servers. SSL also +supports the notion of client certificates that allow the server to validate the identity of a +client. While beyond the scope of this article, the techniques involved are similar to specifying +a custom {@link javax.net.ssl.TrustManager}. +See the discussion about creating a custom {@link javax.net.ssl.KeyManager} in the documentation for +{@link javax.net.ssl.HttpsURLConnection}.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/html/training/training_toc.cs b/docs/html/training/training_toc.cs index 951804683d4ca..79980be6d779e 100644 --- a/docs/html/training/training_toc.cs +++ b/docs/html/training/training_toc.cs @@ -1069,6 +1069,13 @@ "How to perform various tasks and keep your app's data and your user's data secure." >Security Tips + +
  • + Security with HTTPS and SSL +