Asking myself, how to make an REST-like API with Bash<3 ?
socat is used to open port and handle the network calls.
Why ?
Cause I have bug in one project so i needed to think about something else.
More serious, it was also cause i asked myself:
"If I want to write an http api in bash, how-to ?"
From this question, this blog post !
How ?
In Bash, with socat to handle the network calls and the SSL certificate for a most secure transaction.
It is also possible to handle network calls with xinetd or with systemd :'( and it's .socket file.
!! Do not use for production environment !!
To be simple the api will have only one route /hello who will get one parameter /<name>.
The url will be like this => https://ip:port/hello/Operator
Code !
The first thing is to check the request.
Check if the request if correctly formed and if yes, if it is GET request (only GET is allowed here).
function check_request(){
read -r request
echo "> $request" >&2
read -r HTTP_METHOD HTTP_URI HTTP_VER <<< "$request"
# Check if these vars are nonzero string lenght
[ -n "$HTTP_METHOD" ] && [ -n "$HTTP_URI" ] && [ -n "$HTTP_VER" ] \
|| http_code 400 "Bad Request"
# Only 'GET' method accepted
[ "$HTTP_METHOD" = "GET" ] || http_error 405 "Method Not Allowed"
}
> GET / HTTP/1.1
The linux command read is used to catch the http request from stdin and store it in the $request then the first line is sent to stderr aka the socat debug output.
The second read is used to split the $request into three variables: HTTP_METHOD, HTTP_URI, HTTP_VER, check if these variables are non-empty then check if $HTTP_METHOD used is GET.
If the method is not GET, an http error 405 is sent.
The second function called is to log the full http request, here is sent to the socat debug output but it can be sent into file, db etc...
## Log client http header
function log_request_header(){
while read -r request; do
request=${request%%$'\r'}
# when lenght of $request, break or the script stuck
[ -z "$request" ] && break
echo "> $request" >&2
done
echo '>' >&2
}
> Host: localhost:8080
> User-Agent: curl/7.64.0
> Accept: */*
>
A while loop through the rest of the $request to log the header.
When the method is successfully checked and the request logged this is the check of the URI through the $HTTP_URI variable.
## Check if uri exists or 404
function check_uri(){
parsed_uri=`echo "$HTTP_URI" |cut -d'/' -f 2`
declare -a URI
URI+=("hello")
if [[ ! "${URI[*]}" =~ "$parsed_uri" ]]; then
http_error 404 "Not Found"
fi
}
The URI is check by splitting the $HTTP_URI by / and get the second part, this api considere simple URI like /hello, /check, /version.
URI like /check/blah/blah is not supported.
An array URI is set and contains all available URIs of the api.
This line:
if [[ ! "${URI[*]}" =~ "$parsed_uri" ]];
check if $parsed_uri doesn't exists in the array the throw an http error 404.
(This check have to refactored cause it is buggy)
To add an URI just add a line URI+=("URI_HERE_WITHOUT_SLASHES").
If the URI exists, jump to the next function and the main one run(), the one who run functions linked to the URI path.
## Execute local function by name
## from HTTP_URI.
## HTTP_URI = function name
## /hello run hello()
function run(){
function_to_run="${HTTP_URI%/*}"
function_to_run="${function_to_run#/*}"
$function_to_run
}
At this point the $HTTP_URI contains
/hello/Operator
This value is parsed through bash substitution power.
function_to_run="${HTTP_URI%/*}"
/hello
function_to_run="${function_to_run#/*}"
hello
Here the hello() function will be runned, but before I need three other functions, one for the response length, the Content-Lenght header.
An other one, create_request_response() to build the full response with basic header set.
And the the response function, to log and to answer to the client request.
## Get string lenght
function get_lenght(){
lenght="${#1}"
return $lenght
}
The create_request_response() function:
## Declare global array HTTP_RESPONSE_HEADER
## Set default headers
declare -a HTTP_RESPONSE_HEADER
function create_request_response(){
time=$(date +"%a, %d %b %Y %H:%M:%S %Z")
content_lenght="$1"
HTTP_RESPONSE_HEADER+=("Date: $time")
HTTP_RESPONSE_HEADER+=("Server: BashAPI/0.1")
HTTP_RESPONSE_HEADER+=("Last-Modified: $time")
HTTP_RESPONSE_HEADER+=("Content-Lenght: $content_lenght")
HTTP_RESPONSE_HEADER+=("Content-Type: application/json")
}
An array HTTP_RESPONSE_HEADER is declared in the global scope of the script to be able to be called in other functions the I set some default values.
The function take one parameter, $content_lenght. The idea behind this function is to call it with the content_lenght returned by the get_lenght() function.
And the send_response() function.
function send_response(){
echo "< $@" >&2
echo -e "$*"
}
The first line is to log to the socat debug output and the second one is the client response.
Now let's see the hello() function:
## Say hello to the client /hello/client_name
## => /hello/operator
## => Hello operator !
function hello(){
if [[ "$HTTP_URI" =~ /hello/(.*) ]]; then
name="${BASH_REMATCH[1]}"
msg="Hello $name !"
full_response="{\"status\": 200, \"message\": \"$msg\"}"
get_lenght "$full_response"
create_request_response $?
send_response "HTTP/1.1 200 OK"
for head in "${HTTP_RESPONSE_HEADER[@]}"; do
send_response "$head"
done
send_response
send_response $full_response
fi
}
The 'if' statement check if $HTTP_URI matchs /hello/something and capture the value after /hello/ into (.*).
The full match by =~ is stored in the BASH_REMATCH array and the value captured by the parenthesis is stored at the array's index 1 ${BASH_REMATCH[1]} and attribued to the $name variable.
$full_response is a variable who store the response into a json form.
Size of the response is calculate by passing the $full_response to the get_lenght() function, then create_request_response() is called with the return of the get_lenght() function.This is the important part, the answer. Now it is ready to send it.
First the http response, to validate that the server has accepted the client request.
send_response "HTTP/1.1 200 OK"
Then loop through the HTTP_RESPONSE_HEADER to send it to the client:
for head in "${HTTP_RESPONSE_HEADER[@]}"; do
send_response "$head"
done
And the full response:
send_response $full_response
It is also possible to set a check on the client's ip by using the $SOCAT_PEERADDR variable setted by socat itself.
## Check if the client ip is allowed
function check_client(){
declare -a ALLOWED_CLIENTS
ALLOWED_CLIENTS+=("127.0.0.1")
ALLOWED_CLIENTS+=("192.168.14.18")
if [[ ! "${ALLOWED_CLIENTS[*]}" =~ "$SOCAT_PEERADDR" ]]; then
http_error 403 'Unauthorized'
exit 1
fi
}
Output logs:
Client
> GET /hello/Operator HTTP/1.1
> Host: localhost:8082
> User-Agent: curl/7.64.0
> Accept: */*
>
Server
< HTTP/1.1 200 OK
< Date: Mon, 13 Jun 2022 01:24:13 CEST
< Server: BashAPI/0.1
< Last-Modified: Mon, 13 Jun 2022 01:24:13 CEST
< Content-Lenght: 46
< Content-Type: application/json
<
< {"status": 200, "message": "Hello Operator !"}
Full
> GET /hello/Operator HTTP/1.1
> Host: localhost:8082
> User-Agent: curl/7.64.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Mon, 13 Jun 2022 01:24:13 CEST
< Server: BashAPI/0.1
< Last-Modified: Mon, 13 Jun 2022 01:24:13 CEST
< Content-Lenght: 46
< Content-Type: application/json
<
< {"status": 200, "message": "Hello Operator !"}
The full code !
#!/bin/bash
## Send response to client
function send_response(){
echo "< $@" >&2
echo -e "$*"
}
## Get string lenght
function get_lenght(){
lenght="${#1}"
return $lenght
}
## Send json format http error code
function http_error(){
http_code="$1"
http_msg="$2"
send_response "HTTP/1.1 $http_code $http_msg"
send_response
send_response "{\"status\": $http_code}"
exit 1
}
## Check if the client ip is allowed
function check_client(){
declare -a ALLOWED_CLIENTS
ALLOWED_CLIENTS+=("127.0.0.1")
ALLOWED_CLIENTS+=("192.168.14.18")
#allowed='/srv/code/bashapi/socat/allowed.txt'
if [[ ! "${ALLOWED_CLIENTS[*]}" =~ "$SOCAT_PEERADDR" ]]; then
http_error 403 'Unauthorized'
exit 1
fi
}
## Check if request has correct form and if it is a GET method
function check_request(){
read -r request
echo "> $request" >&2
read -r HTTP_METHOD HTTP_URI HTTP_VER <<< "$request"
# Check if these vars are nonzero string lenght
[ -n "$HTTP_METHOD" ] && [ -n "$HTTP_URI" ] && [ -n "$HTTP_VER" ] \
|| http_code 400 "Bad Request"
# Only 'GET' method accepted
[ "$HTTP_METHOD" = "GET" ] || http_error 405 "Method Not Allowed"
}
## Log client http header
function log_request_header(){
while read -r request; do
request=${request%%$'\r'}
# when lenght of $request, break or the script stuck
[ -z "$request" ] && break
echo "> $request" >&2
done
echo '>' >&2
}
## Declare global array HTTP_RESPONSE_HEADER
## Set default headers
declare -a HTTP_RESPONSE_HEADER
function create_request_response(){
time=$(date +"%a, %d %b %Y %H:%M:%S %Z")
content_lenght="$1"
HTTP_RESPONSE_HEADER+=("Date: $time")
HTTP_RESPONSE_HEADER+=("Server: BashAPI/0.1")
HTTP_RESPONSE_HEADER+=("Last-Modified: $time")
HTTP_RESPONSE_HEADER+=("Content-Lenght: $content_lenght")
HTTP_RESPONSE_HEADER+=("Content-Type: application/json")
}
## Check if uri exists or 404
function check_uri(){
parsed_uri=`echo "$HTTP_URI" |cut -d'/' -f 2`
declare -a URI
URI+=("hello")
if [[ ! "${URI[*]}" =~ "$parsed_uri" ]]; then
http_error 404 "Not Found"
fi
}
## Say hello to the client /hello/client_name
## => /hello/operator
## => Hello operator !
function hello(){
if [[ "$HTTP_URI" =~ /hello/(.*) ]]; then
name="${BASH_REMATCH[1]}"
msg="Hello $name !"
full_response="{\"status\": 200, \"message\": \"$msg\"}"
get_lenght "$full_response"
create_request_response $?
send_response "HTTP/1.1 200 OK"
for head in "${HTTP_RESPONSE_HEADER[@]}"; do
send_response "$head"
done
send_response
#get_lenght
send_response $full_response
fi
}
## Execute local function by name
## from HTTP_URI.
## HTTP_URI = function name
## /hello run hello()
function run(){
function_to_run="${HTTP_URI%/*}"
function_to_run="${function_to_run#/*}"
$function_to_run
}
check_client
check_request
log_request_header
check_uri
run
To run it on non-secure:
socat tcp-listen:8080,fork EXEC:./bashapi.sh
Call it:
curl -s localhost:8080/hello/Operator | jq .
Response:
{
"status": 200,
"message": "Hello Operator !"
}
And to run it with SSL, first create a certificate:
# openssl req -new -newkey rsa:4096 -days 99999 -nodes -x509 \
-subj "/C=FR/ST=PA/L=PA/O=BashAPI/CN=BashAPI.run" \
-keyout bashapi.run.key -out bashapi.run.crt
# cat bashapi.run.key bashapi.run.crt > bashapi.run.pem
Then, run socat with the certificate:
socat openssl-listen:8080,fork,cert=./bashapi.run.pem,verify=0 EXEC:./bashapi.sh
Call it:
curl -sk https://localhost:8080/hello/Operator | jq .
Response:
{
"status": 200,
"message": "Hello Operator !"
}
Was just a little PoC to see how to create a simple rest-like api with bash.
The only missing thing to bash is the ability to handle socket listening, thanks to it's ability to handle any program in the system like socat <3