Set up Apache Httpd 2.2 to cache static contents from Weblogic / Tomcat / Jboss backend

Configuration:

Add below to conf.d/backend.conf

<IfModule mod_cache.c>
<IfModule mod_disk_cache.c>
CacheRoot “/var/cache/mod_proxy/”
CacheEnable disk “/”
CacheDirLevels 2
CacheDirLength 1

CacheLock on
CacheLockPath “/tmp/mod_cache-lock”
CacheLockMaxAge 5
</IfModule>

# These only valid for Apache httpd 2.4
#CacheQuickHandler off
#CacheHeader on
#CustomLog “logs/cached-requests.log” common env=cache-hit
#CustomLog “logs/uncached-requests.log” common env=cache-miss
#CustomLog “logs/revalidated-requests.log” common env=cache-revalidate
#CustomLog “logs/invalidated-requests.log” common env=cache-invalidate

</IfModule>

 

How to check:

1) Check access logs for Apache as well as the backend

2) Confirm that there are files in CacheRoot

#ls -R /var/cache/mod_proxy

 

References:

https://httpd.apache.org/docs/2.2/caching.html

https://httpd.apache.org/docs/current/mod/mod_cache.html

 

Advertisements

Securing Tomcat – Part 3

Vulnerability: Allowed HTTP Methods: OPTIONS, PUT, DELETE, TRACE

Remediation:

– add in web.xml
<security-constraint>
<web-resource-collection>
<web-resource-name>restricted methods</web-resource-name>
<url-pattern>/*</url-pattern>
<http-method>PUT</http-method>
<http-method>DELETE</http-method>
<http-method>OPTIONS</http-method>
<http-method>TRACE</http-method>
</web-resource-collection>
<auth-constraint />
</security-constraint>

How to check:
for method in OPTIONS GET HEAD POST PUT DELETE TRACE CONNECT ; do
echo -e “\n\nTrying $method\n\n”
echo -e “$method / HTTP/1.1\nHost: localhost\nConnection: close\n\n” | nc localhost 8180 | head -1
sleep 2
done

Note: Tomcat 8 is not vulnerable since JSP 2.3 specs only permit GET POST or HEAD.

Securing Tomcat 8 – Part 2

Vulnerability: Clickjacking

How to check: 

Create a page that call the site in (should fail to display in the frame)

<html>
<head>
<title>Clickjack test page</title>
</head>
<body>
<p>Website is vulnerable to clickjacking if you can see it below!</p>
http://www.target.site
</body>
</html>

Remediation: 

Create in $TOMCAT_BASE/lib/org/owasp/filters/ClickjackFilter.class

public class ClickjackFilter implements Filter
{

private String mode = “DENY”;

/**
* Add X-FRAME-OPTIONS response header to tell IE8 (and any other browsers who
* decide to implement) not to display this content in a frame. For details, please
* refer to http://blogs.msdn.com/sdl/archive/2009/02/05/clickjacking-defense-in-ie8.aspx.
*/
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
HttpServletResponse res = (HttpServletResponse)response;
//If you have Tomcat 5 or 6, there is a known bug using this code. You must have the doFilter first:
chain.doFilter(request, response);
res.addHeader(“X-FRAME-OPTIONS”, mode );
//Otherwise use this:
//res.addHeader(“X-FRAME-OPTIONS”, mode );
//chain.doFilter(request, response);

}

public void destroy() {
}

public void init(FilterConfig filterConfig) {
String configMode = filterConfig.getInitParameter(“mode”);
if ( configMode != null ) {
mode = configMode;
}
}
}

Add to web.xml

<filter>
<filter-name>ClickjackFilterDeny</filter-name>
<filter-class>org.owasp.filters.ClickjackFilter</filter-class>
<init-param>
<param-name>mode</param-name>
<param-value>DENY</param-value>
</init-param>
</filter>

<filter-mapping>
<filter-name>ClickjackFilterDeny</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

Verification: In addition to <iframe> check above, can also verify that X-FRAME-OPTIONS cookie is present.

[root@localhost webapps]# telnet localhost 8180
Trying 127.0.0.1…
Connected to localhost.
Escape character is ‘^]’.
GET http://localhost:8180/index.jsp HTTP/1.1

HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=49A0FE6E28D5C04362CC830ECB4732F7; Path=/; HttpOnly
X-FRAME-OPTIONS: DENY
Content-Type: text/html;charset=ISO-8859-1
Content-Length: 6
Date: Thu, 10 Nov 2016 07:02:16 GMT
Server: MyServer

Hello

^C
Connection closed by foreign host.

References:

https://www.owasp.org/index.php/Clickjacking_Defense_Cheat_Sheet

Notes:

If using supported version of Tomcat, only need to set filter in web.xml without requiring to do all above

<filter>
<filter-name>httpHeaderSecurity</filter-name>
<filter-class>org.apache.catalina.filters.HttpHeaderSecurityFilter</filter-class>
<init-param>
<param-name>antiClickJackingOption</param-name>
<param-value>SAMEORIGIN</param-value>
</init-param>
</filter>

<filter-mapping>
<filter-name>httpHeaderSecurity</filter-name>
<url-pattern>/*</url-pattern>
<dispatcher>REQUEST</dispatcher>
</filter-mapping>

If the tomcat in use does not support it, will see below exception

09-Nov-2016 11:44:00.655 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.filterStart Exception starting filter httpHeaderSecurity
java.lang.ClassNotFoundException: org.apache.catalina.filters.HttpHeaderSecurityFilter
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

Securing Tomcat 8 – Part 1

Vulnerability: Information disclosure in server header and error page

How to check: 

[root@localhost conf]# telnet localhost 8180
Trying 127.0.0.1…
Connected to localhost.
Escape character is ‘^]’.
GET http://localhost:8180/dummy.jsp HTTP/1.1

HTTP/1.1 404 Not Found
Server: Apache-Coyote/1.1
Content-Type: text/html;charset=utf-8
Content-Language: en
Content-Length: 1014
Date: Wed, 09 Nov 2016 07:22:06 GMT

<!DOCTYPE html><html><head><title>Apache Tomcat/8.0.18 – Error report</title><style type=”text/css”>H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}.line {height: 1px; background-color: #525D76; border: none;}</style> </head><body><h1>HTTP Status 404 – /dummy.jsp</h1>

<p><b>type</b> Status report</p><p><b>message</b> <u>/dummy.jsp</u></p><p><b>description</b> <u>The requested resource is not available.</u></p><hr class=”line”><h3>Apache Tomcat/8.0.18</h3></body></html>
^C
Connection closed by foreign host.

Remediation:

Edit server.xml
<Connector port=”8180″ …
server=”MyServer”

<Valve className=”org.apache.catalina.valves.ErrorReportValve”
showReport=”false” showServerInfo=”false” />

 

[root@localhost conf]# telnet localhost 8180
Trying 127.0.0.1…
Connected to localhost.
Escape character is ‘^]’.
GET http://localhost:8180/dummy.jsp HTTP/1.1

HTTP/1.1 404 Not Found
Content-Type: text/html;charset=utf-8
Content-Language: en
Content-Length: 118
Date: Wed, 09 Nov 2016 07:28:04 GMT
Server: MyServer

<!DOCTYPE html><html><head><title>Error report</title></head><body><h1>HTTP Status 404 – /dummy.jsp</h1></body></html>
^C
Connection closed by foreign host.

References:
https://www.owasp.org/index.php/Securing_tomcat
http://www.ibm.com/developerworks/library/se-banner/

 

Apache Httpd 2.2.x proxying request to Tomcat 8

Add below entries to /etc/httpd/conf.d/tomcat.conf:

SetEnv force-proxy-request-1.0 1
SetEnv proxy-nokeepalive 1
SetEnv proxy-initial-not-pooled 1
RequestHeader unset Expect early

ProxyPass “/abc” “balancer://mycluster/”
<Proxy balancer://mycluster>
ProxySet failonstatus=502,503 maxattempts=100
BalancerMember ajp://host1:9999 timeout=10 retry=10 ping=10 disablereuse=on keepalive=on
BalancerMember ajp://host2:9999 timeout=10 retry=10 ping=10 disablereuse=on keepalive=on
BalancerMember ajp://host3:9999 timeout=10 retry=10 ping=10 disablereuse=on keepalive=on
</Proxy>

 

Some explanation:

SetEnv force-proxy-request-1.0 1
SetEnv proxy-nokeepalive 1
– Force the request to use HTTP/1.0 with no keepalive.

SetEnv proxy-initial-not-pooled 1
– If this variable is set, no pooled connection will be reused if the client request is the initial request on the frontend connection. This avoids the “proxy: error reading status line from remote server” error message caused by the race condition that the backend server closed the pooled connection after the connection check by the proxy and before data sent by the proxy reached the backend. It has to be kept in mind that setting this variable downgrades performance, especially with HTTP/1.0 clients.

RequestHeader unset Expect early
– The issue is that some clients set the Expect header and only send the request headers before a PUT or POST of data. This allows the server to respond with errors/redirects/security violations prior to the client sending the request body (PUT or POST data). Apparently some clients does not wait until it gets a response and just pushes out the body of the request, which results in the 417 error.
http://stackoverflow.com/questions/3889574/apache-and-mod-proxy-not-handling-http-100-continue-from-client-http-417

 

ProxySet failonstatus=502,503 maxattempts=10
failonstatus – A single or comma-separated list of HTTP status codes. Will force the worker into error state when the backend returns any status code in the list
maxattempts – Maximum number of failover attempts before giving up.

BalancerMember ajp://host1:9999 timeout=10 retry=10 ping=10 disablereuse=on keepalive=on
timeout – maximum time to wait for a free worker. The default is to not wait.
retry – Connection pool worker retry timeout in seconds. If the connection pool worker to the backend server is in the error state, Apache httpd will not forward any requests to that server until the timeout expires. This enables to shut down the backend server for maintenance and bring it back online later. A value of 0 means always retry workers in an error state with no timeout.
ping – Delay in seconds to wait for the reply of “ping test” to the backend
disablereuse – force mod_proxy to immediately close a connection to the backend after being used
keepalive – This parameter should be used when you have a firewall between your Apache httpd and the backend server, which tends to drop inactive connections. This flag will tell the Operating System to send KEEP_ALIVE messages on inactive connections and thus prevent the firewall from dropping the connection

References:

https://bz.apache.org/bugzilla/show_bug.cgi?format=multiple&id=57520

https://httpd.apache.org/docs/current/mod/mod_proxy.html

Systemctl for Tomcat in RedHat Enterprise Linux 7

If tomcat requires parameters that have been set in .bash_profile, below are steps to configure systemctl:

1) Create files with below content (check ownership and permission)

#!/bin/sh
#/appl/myservice/scripts/start.sh

SERVICE=’myservice’
SERVICE_USER=’tc_myservice’

START_TOMCAT=”/opt/tomcat.${SERVICE}/bin/startup.sh”

if [ X”$USER” = X”$SERVICE_USER” ]; then
${START_TOMCAT}
else
su – ${SERVICE_USER} -c ${START_TOMCAT}
fi
#!/bin/sh
#/appl/myservice/scripts/stop.sh

SERVICE=’myservice’
SERVICE_USER=’tc_myservice’

STOP_TOMCAT=”/opt/tomcat.${SERVICE}/bin/shutdown.sh”

if [ X”$USER” = X”$SERVICE_USER” ]; then
${STOP_TOMCAT}
else
su – ${SERVICE_USER} -c ${STOP_TOMCAT}
fi

 

2) Create file /etc/systemd/system/tomcat.myservice.service

[Unit]
Description=Tomcat process for myservice
After=multi-user.target

[Service]
Type=forking
User=tc_myservice
Group=tc_myservice
ExecStartPre=/bin/bash –login -c ‘env > /tmp/.myservice-environment-file’
ExecStart=/appl/myservice/scripts/start.sh
ExecStop=/appl/myservice/scripts/stop.sh
EnvironmentFile=-/tmp/.myservice-environment-file
Restart=no

[Install]
WantedBy=multi-user.target

 

3) Enable the service from systemctl
systemctl enable tomcat.myservice
systemctl daemon-reload

 

4) Test
systemctl start tomcat.myservice
systemctl status tomcat.myservice -l

systemctl stop tomcat.myservice
systemctl status tomcat.myservice -l

Simple SSL Perfomance Comparison

Command used to test:

# openssl s_time -connect localhost:$PORT -new

Tomcat APR/Native:
2011 connections in 10.67s; 188.47 connections/user sec, bytes read 0
2011 connections in 31 real seconds, 0 bytes read per connection

JBoss Java connector:
460 connections in 11.01s; 41.78 connections/user sec, bytes read 0
460 connections in 31 real seconds, 0 bytes read per connection

Apache:
2753 connections in 14.43s; 190.78 connections/user sec, bytes read 0
2753 connections in 31 real seconds, 0 bytes read per connection

Weblogic:
797 connections in 3.81s; 209.19 connections/user sec, bytes read 0
797 connections in 31 real seconds, 0 bytes read per connection

Conclusion:

  1. Using APR/Native is better for performance.
  2. Weblogic is quite OK, performance better than JBoss, considering that it is using JDK 6 (will try with JDK 7 & 8 later).

Updates (26 Oct 2016)

Weblogic + JDK 7
1103 connections in 2.86s; 385.66 connections/user sec, bytes read 0
1103 connections in 31 real seconds, 0 bytes read per connection

Weblogic + JDK 8
889 connections in 4.27s; 208.20 connections/user sec, bytes read 0
889 connections in 31 real seconds, 0 bytes read per connection

Testing Tomcat 8 SSL using testssl.sh

Configuration:

Apache Tomcat 8.0.18 + JDK 1.8.0_92

APR/Native SSL Connector defined in server.xml:

<Connector
protocol=”org.apache.coyote.http11.Http11AprProtocol”
port=”8443″ maxThreads=”100″
scheme=”https” secure=”true” SSLEnabled=”true”
SSLCertificateFile=”conf/certs/cert.pem”
SSLCertificateKeyFile=”conf/certs/key.pem”
SSLVerifyClient=”optional” SSLProtocol=”TLSv1.1+TLSv1.2″
SSLHonorCipherOrder=”true” SSLCipherSuite=”HIGH:!aNULL:!eNULL:!EXPORT:!3DES:!DES:!RC4:!MD5:!kRSA”/>

Result: ALL OK!

[root@localhost testssl]# ./testssl.sh –quiet localhost:8443

No engine or GOST support via engine with your /usr/bin/openssl

Testing now (2016-10-24 15:39) —> 127.0.0.1:8443 (localhost) <—

A record via /etc/hosts
rDNS (127.0.0.1): localhost.
Service detected: HTTP
–> Testing protocols (via sockets except TLS 1.2 and SPDY/NPN)

SSLv2 not offered (OK)
SSLv3 not offered (OK)
TLS 1 not offered
TLS 1.1 offered
TLS 1.2 offered (OK)
SPDY/NPN not offered

–> Testing ~standard cipher lists

Null Ciphers not offered (OK)
Anonymous NULL Ciphers not offered (OK)
Anonymous DH Ciphers not offered (OK)
40 Bit encryption not offered (OK)
56 Bit encryption Local problem: No 56 Bit encryption configured in /usr/bin/openssl
Export Ciphers (general) not offered (OK)
Low (<=64 Bit) not offered (OK)
DES Ciphers not offered (OK)
Medium grade encryption not offered (OK)
Triple DES Ciphers not offered (OK)
High grade encryption offered (OK)

–> Testing (perfect) forward secrecy, (P)FS — omitting 3DES, RC4 and Null Encryption here

PFS is offered (OK) ECDHE-RSA-AES256-GCM-SHA384 DHE-RSA-AES256-GCM-SHA384 DHE-RSA-AES256-SHA256 DHE-RSA-AES256-SHA DHE-RSA-CAMELLIA256-SHA ECDHE-RSA-AES256-SHA384 ECDHE-RSA-AES256-SHA ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA256 DHE-RSA-AES128-GCM-SHA256 DHE-RSA-AES128-SHA256 DHE-RSA-AES128-SHA DHE-RSA-CAMELLIA128-SHA ECDHE-RSA-AES128-SHA

–> Testing server preferences

Has server cipher order? yes (OK)
Negotiated protocol TLSv1.2
Negotiated cipher ECDHE-RSA-AES256-GCM-SHA384, 256 bit ECDH
Cipher order
TLSv1.1: ECDHE-RSA-AES256-SHA DHE-RSA-AES256-SHA DHE-RSA-CAMELLIA256-SHA ECDHE-RSA-AES128-SHA DHE-RSA-AES128-SHA DHE-RSA-CAMELLIA128-SHA
TLSv1.2: ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-SHA384 ECDHE-RSA-AES256-SHA DHE-RSA-AES256-GCM-SHA384 DHE-RSA-AES256-SHA256 DHE-RSA-AES256-SHA DHE-RSA-CAMELLIA256-SHA ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA256 ECDHE-RSA-AES128-SHA DHE-RSA-AES128-GCM-SHA256 DHE-RSA-AES128-SHA256 DHE-RSA-AES128-SHA DHE-RSA-CAMELLIA128-SHA

–> Testing server defaults (Server Hello)

TLS server extensions renegotiation info, EC point formats, session ticket, heartbeat
Session Tickets RFC 5077 300 seconds
Server key size 2048 bit
Signature Algorithm SHA1 with RSA
Fingerprint / Serial SHA1 8AB61B39A292A0826D3C744F229843F58EFB6B13 / C95DEAE861AF547F
SHA256 B6F2DF7EDABEA1249972AEA88A7EE8AD9DE40326DF013EA71FD9AF874CAB8355
Common Name (CN) localhost (works w/o SNI)
subjectAltName (SAN) —
Issuer localhost (MyCompany from MY)
EV cert (experimental) no
Certificate Expiration >= 60 days (2016-10-24 15:09 –> 2026-10-22 15:09 +0800)
# of certificates provided 1
Chain of trust (experim.) Your /usr/bin/openssl is too old, needed is version >=1.0.2
Certificate Revocation List —
OCSP URI —
OCSP stapling not offered
TLS clock skew -1 sec from localtime
–> Testing HTTP header response @ “/”

HTTP Status Code 404 Not Found (Hint: supply a path which doesn’t give a “404 Not Found”)
HTTP clock skew 0 sec from localtime
Strict Transport Security —
Public Key Pinning —
Server banner Apache-Coyote/1.1
Application banner —
Cookie(s) (none issued at “/”)
Security headers —
Reverse Proxy banner —
–> Testing vulnerabilities

Heartbleed (CVE-2014-0160) not vulnerable (OK) (timed out)
CCS (CVE-2014-0224) not vulnerable (OK)
Secure Renegotiation (CVE-2009-3555) not vulnerable (OK)
Secure Client-Initiated Renegotiation not vulnerable (OK)
CRIME, TLS (CVE-2012-4929) not vulnerable (OK)
BREACH (CVE-2013-3587) no HTTP compression (OK) (only supplied “/” tested)
POODLE, SSL (CVE-2014-3566) not vulnerable (OK)
TLS_FALLBACK_SCSV (RFC 7507), experim. Downgrade attack prevention supported (OK)
FREAK (CVE-2015-0204) not vulnerable (OK) (tested with 4/9 ciphers)
LOGJAM (CVE-2015-4000), experimental not vulnerable (OK) (tested w/ 2/4 ciphers only!), common primes not checked. See below for any DH ciphers + bit size
BEAST (CVE-2011-3389) no SSL3 or TLS1
RC4 (CVE-2013-2566, CVE-2015-2808) no RC4 ciphers detected (OK)
–> Testing all locally available 121 ciphers against the server, ordered by encryption strength

Hexcode Cipher Suite Name (OpenSSL) KeyExch. Encryption Bits Cipher Suite Name (RFC)
———————————————————————————————————————–
xc030 ECDHE-RSA-AES256-GCM-SHA384 ECDH 256 AESGCM 256 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
xc028 ECDHE-RSA-AES256-SHA384 ECDH 256 AES 256 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
xc014 ECDHE-RSA-AES256-SHA ECDH 256 AES 256 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
x9f DHE-RSA-AES256-GCM-SHA384 DH 1024 AESGCM 256 TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
x6b DHE-RSA-AES256-SHA256 DH 1024 AES 256 TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
x39 DHE-RSA-AES256-SHA DH 1024 AES 256 TLS_DHE_RSA_WITH_AES_256_CBC_SHA
x88 DHE-RSA-CAMELLIA256-SHA DH 1024 Camellia 256 TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA
xc02f ECDHE-RSA-AES128-GCM-SHA256 ECDH 256 AESGCM 128 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
xc027 ECDHE-RSA-AES128-SHA256 ECDH 256 AES 128 TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
xc013 ECDHE-RSA-AES128-SHA ECDH 256 AES 128 TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
x9e DHE-RSA-AES128-GCM-SHA256 DH 1024 AESGCM 128 TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
x67 DHE-RSA-AES128-SHA256 DH 1024 AES 128 TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
x33 DHE-RSA-AES128-SHA DH 1024 AES 128 TLS_DHE_RSA_WITH_AES_128_CBC_SHA
x45 DHE-RSA-CAMELLIA128-SHA DH 1024 Camellia 128 TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA
Done now (2016-10-24 15:40) —> 127.0.0.1:8443 (localhost) <—

References:

https://tomcat.apache.org/tomcat-8.0-doc/config/http.html