Sunday, May 28, 2017

Solaris' pargs, penv, pfiles, pmap, pstack, and pwdx on macOS

Three weeks ago I posted Solaris' pargs, penv, pfiles, pmap, and pstack on GNU/Linux and since I am also a Mac user, I thought it could be a good idea to have those commands also on macOS. Here we go ...

function pargs()  { L=$(ps ww $1 | tail -1); echo ${L:27}; }
function penv()   { L=$(pargs $1); C=${#L}; L=$(ps wwe $1 | tail -1); L=${L:27}; echo ${L:$C} | tr ' ' '\n'; }
function pfiles() { lsof -p $1; }
function pmap()   { vmmap $1; }
function pstack() { echo "thread backtrace all" | lldb -p $1; }
function pwdx()   { L=$(lsof -a  -d cwd -p $1 | tail -1); echo /${L#*/}; }

Since there is no access to a proc file system on macOS (at least not by default), both pargs and penv call the ps command and pmap calls vmmap. Furthermore pwdx calls lsof with the current working directory descriptor request in order to get the required info. Since "ps wwe" returns not only the environment variables for the given process but also the program arguments on macOS, we need to strip the program arguments from the output. This has been done by calling pargs, determining the length of that output and cutting that length from the string again before we pass it to the tr command that gives us an environment variable for each line. For blog purposes I have shortened the variable names, L stands for line and C for count.

References:
http://wiki.bash-hackers.org/syntax/pe
http://yongsun.me/2009/01/tips-the-equivalents-of-ldd1-and-pmap1-on-mac-os-x/

Sunday, May 7, 2017

Solaris' pargs, penv, pfiles, pmap, and pstack on GNU/Linux

I really like the p-commands on Sun/Oracle Solaris and I miss those on GNU/Linux. Therefore I have gathered/created rudimental equivalent one-liners that work on a PID.

I have added the following functions to my ~/.bash_aliases file that is being sourced by my ~/.bashrc. pargs, penv, and pmap are gathering the proc file system, while pfiles and pstack are calling lsof resp. gdb.

function pargs() { cat /proc/$1/cmdline | tr '\0' ' ' | sed 's/ $/\n/g'; }
function penv() { cat /proc/$1/environ | tr '\0' '\n'; }
function pfiles() { lsof -p $1; }

function pmap() { cat /proc/$1/maps; }
function pstack() { sudo gdb --pid=$1 --batch -ex "thread apply all bt"; }


Note that due to a kernel hardening (ptrace protection) on Ubuntu 10.10 and later you need to call gdb under the control of sudo, or alternatively modify the ptrace_scope property. See also https://wiki.ubuntu.com/SecurityTeam/Roadmap/KernelHardening#ptrace_Protection and https://askubuntu.com/questions/41629/after-upgrade-gdb-wont-attach-to-process


Friday, February 10, 2017

How the NumericalChameleon Installer obtains the latest JRE for Windows

Today I will explain how the NumericalChameleon installer obtains the latest Java Runtime Environment (JRE) for Microsoft Windows.

The NumericalChameleon (http://numericalchameleon.net) is written in Java and it relies on a JRE that is installed on your system. The NumericalChameleon installer checks at first whether a JRE is installed. If there is one, everything is fine and the installer will continue with a normal installation. If there is none, the installer downloads the latest JRE offline installer from Oracle and launches it. The offline installer installs the JRE on your system and once it is installed, it will give back the control to the NumericalChameleon installer that continues with the installation until the NumericalChameleon software package is installed as well.

Below you find a screenshot of the NumericalChameleon installer, localized in German, running on Windows 10 x64 while downloading the latest JRE offline installer:



Since the actual locations of the JRE offline exe installers are different for each Java version, and those locations are both unpredictable and volatile, it is important for the NumericalChameleon installer to rely on well known static URIs, because the installer binary cannot be changed/patched anymore once it is deployed on the web.

Those well known URIs are redirects in the .htaccess file on my Apache server actually, and the redirects are being updated every day. I create those redirects by parsing the website that has the locations of the Windows JRE offline installers.

$ cat bin/update_htaccess
#!/bin/bash
HTACCESS="$HOME/numericalchameleon/.htaccess"
STATIC="$HOME/numericalchameleon/.htaccess.static"

cat $STATIC > $HTACCESS

# the locations of the JRE offline installers
URL="https://www.java.com/en/download/manual.jsp"
CONTENT=$(curl -s "$URL")

IDENTIFIER="Windows Offline</a>"
if [[ "$CONTENT" =~ BundleId=([^\"]+)\"\>[[:space:]]*"$IDENTIFIER" ]]; then
    printf "Redirect /get_java_win32bit http://javadl.oracle.com/webapps/download/AutoDL?BundleId=%s\n" ${BASH_REMATCH[1]} >> $HTACCESS
fi

IDENTIFIER="Windows Offline (64-bit)</a>"
if [[ "$CONTENT" =~ BundleId=([^\"]+)\"\>[[:space:]]*"$IDENTIFIER" ]]; then
    printf "Redirect /get_java_win64bit http://javadl.oracle.com/webapps/download/AutoDL?BundleId=%s\n" ${BASH_REMATCH[1]} >> $HTACCESS
fi

The .htaccess.static file just contains entries that never change. The update_htaccess script runs by cron periodically on the server in order to have entries that are up to date.

$ crontab -l | tail -1

9 3 * * * cd bin; ./update_htaccess

After the cron was running, updated redirects can be found in the .htaccess file. In the example below the redirects are from Java 8u121:

$ tail -2 .htaccess

Redirect /get_java_win32bit http://javadl.oracle.com/webapps/download/AutoDL?BundleId=218831_e9e7ea248e2c4826b92b3f075a80e441
Redirect /get_java_win64bit http://javadl.oracle.com/webapps/download/AutoDL?BundleId=218833_e9e7ea248e2c4826b92b3f075a80e441

The installer can now rely on the fixed addresses below, dependent whether a 32 bit or a 64 bit system has been found:
Now you know how the NumericalChameleon installer gets the latest JRE on Windows.

Hint: if the approach above should ever fail and the redirects are not being created, the current installer will fail with a 404. In that case you can simply install the JRE manually before you start the NumericalChameleon installer. In that case the installer won't even go to the internet, because the condition is met already that a JRE has to be installed.