05 Dec 2019

nc + curl

The Goal

Let’s play with netcat and curl

Setup

We open a terminal window and copy-paste this command :

We use the -l option to specify that nc should listen for an incoming connection.
We use the -p <port> option to specifies the source port nc should use.
We use the -q <seconds> option to recover the prompt. After EOF on stdin, wait the specified number of seconds and then quit.

$ while true; do 
    echo -e "HTTP/1.1 200 OK\n\n$(date +'%T')" \
        | nc -l -p 3000 -q 0;
done

Then open a new terminal window to send curl requests.

Send GET requests

The simplest call :

$ curl 'http://localhost:3000'

# the output in the other terminal window
GET / HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.58.0
Accept: */*

A little more elaborate :

$ curl 'http://localhost:3000/path/to/something?a=1&b=2'

# the output in the other terminal window
GET /path/to/something?a=1&b=2 HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.58.0
Accept: */*

Send POST requests

We use the -d, –data option to send the values.
We can note that Content-Type entity header is defined to application/x-www-form-urlencoded.
If the –data option is defined, the request is automatically defined as POST. It is not necessary to declare : --request POST.

$ curl --data 'c=3&d=4' --data 'e=5' 'http://localhost:3000?a=1&b=2'

# the output in the other terminal window
POST /?a=1&b=2 HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.58.0
Accept: */*
Content-Length: 11
Content-Type: application/x-www-form-urlencoded

c=3&d=4&e=5

Let’s see the difference between the –data-urlencode and -d, –data options :

$ curl --data 'abc=1 2 3' --data-urlencode 'efg=4 5 6' 'http://localhost:3000'

# look at the values
POST / HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.58.0
Accept: */*
Content-Length: 23
Content-Type: application/x-www-form-urlencoded

abc=1 2 3&efg=4%205%206

If we want send JSON data, we must define the -H, –header header/@file option :

# without header
$ curl --data '{"a":1, b:true}' 'http://localhost:3000'

# the output say `application/x-www-form-urlencoded`
POST / HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.58.0
Accept: */*
Content-Length: 15
Content-Type: application/x-www-form-urlencoded

{"a":1, b:true}

# with header
$ curl --header 'Content-Type: application/json' --data '{"a":1, b:true}' 'http://localhost:3000'

# the output say `application/json`
POST / HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.58.0
Accept: */*
Content-Type: application/json
Content-Length: 15

{"a":1, b:true}

Send binary data

Let’s create a binary file from a base64 string :

It’s a 100 width x 10 height with red color PNG file.

$ echo iVBORw0KGgoAAAANSUhEUgAAAGQAAAAKCAIAAADNfmwpAAAAKElEQVRIx+3QMQEA\
AAgDILV/51nBzwci0CmuRoEsWbJkyZKlQJasbws3SQETbGZtsAAAAABJRU5ErkJggg==\
    | base64 --decode > red.png

# we can see the binary data
$ cat red.png
�PNG
▒
IHDRd
�~l)(IDATH��� ��Y��"�)�F�,Y�dɒ�@��o
                                   7Ilfm�IEND�B`�

Let’s make a first send :

We use the -F, –form <name=content> option to send the file.
We can note that Content-Type entity header is defined to multipart/form-data.

$ curl --form 'img=@red.png' 'http://localhost:3000'

# the output say `multipart/form-data`
POST / HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.58.0
Accept: */*
Content-Length: 280
Content-Type: multipart/form-data; boundary=------------------------3931a3dc001e9202

--------------------------3931a3dc001e9202
Content-Disposition: form-data; name="img"; filename="red.png"
Content-Type: image/png

�PNG
▒
IHDRd
�~l)(IDATH��� ��Y��"�)�F�,Y�dɒ�@��o
                                   7Ilfm�IEND�B`�
--------------------------3931a3dc001e9202--

A variation with several values sent :

$ curl --form 'a=1' --form 'img=@red.png' --form 'b=2' 'http://localhost:3000'

# the output say `multipart/form-data`
POST / HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.58.0
Accept: */*
Content-Length: 462
Content-Type: multipart/form-data; boundary=------------------------7c95feec758938d0

--------------------------7c95feec758938d0
Content-Disposition: form-data; name="a"

1
--------------------------7c95feec758938d0
Content-Disposition: form-data; name="img"; filename="red.png"
Content-Type: image/png

�PNG
▒
IHDRd
�~l)(IDATH��� ��Y��"�)�F�,Y�dɒ�@��o
                                   7Ilfm�IEND�B`�
--------------------------7c95feec758938d0
Content-Disposition: form-data; name="b"

2
--------------------------7c95feec758938d0--

Let’s make a second send :

We use the –data-binary option to send the file.
We must define the -H, –header header/@file option to Content-Type: image/png.

$ curl --header 'Content-Type: image/png' --data-binary '@red.png' 'http://localhost:3000'

# the output say `image/png`
POST / HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.58.0
Accept: */*
Content-Type: image/png
Content-Length: 97

�PNG
▒
IHDRd
�~l)(IDATH��� ��Y��"�)�F�,Y�dɒ�@��o
                                   7Ilfm�IEND�B`�

Let’s make a third send :

We use base64 to send the binary data as a string.
We send it in a JSON format.
We use --data @-. If you start the data with the letter @, the rest should be a file name to read the data from, or - if you want curl to read the data from stdin.

$ (echo -n '{"image": "'; base64 --wrap 0 red.png; echo '"}') \
    | curl --header "Content-Type: application/json" \
        --data @- 'http://localhost:3000'

# the output say `application/json`
POST / HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.58.0
Accept: */*
Content-Type: application/json
Content-Length: 145

{"image": "iVBORw0KGgoAAAANSUhEUgAAAGQAAAAKCAIAAADNfmwpAAAAKElEQVRIx+3QMQEAAAgDILV/51nBzwci0CmuRoEsWbJkyZKlQJasbws3SQETbGZtsAAAAABJRU5ErkJggg=="}

Let’s make an other send for the fun :

Imagine that we already have a JSON data string.
We will add the base64 data of the image with jq.

$ json='{"a":1}'
$ str=$(base64 --wrap 0 red.png)
$ echo "$json" \
    | jq --arg image "$str" '. + {image: $image}' \
    | curl --header "Content-Type: application/json" \
        --data @- 'http://localhost:3000'

# the output say `application/json`
POST / HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.58.0
Accept: */*
Content-Type: application/json
Content-Length: 156

{  "a": 1,  "image": "iVBORw0KGgoAAAANSUhEUgAAAGQAAAAKCAIAAADNfmwpAAAAKElEQVRIx+3QMQEAAAgDILV/51nBzwci0CmuRoEsWbJkyZKlQJasbws3SQETbGZtsAAAAABJRU5ErkJggg=="}

A variation :

You can use the <<< operator to redirect stdin from a literal string.

$ json='{"a":1}'
$ str=$(base64 --wrap 0 red.png)
$ jq --arg image "$str" '. + {image: $image}' <<< "$json" \
    | curl --header "Content-Type: application/json" \
        --data @- 'http://localhost:3000'

curl jq nc note