From ae578a29c24e19d372b5161463c6d2571af292b5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?P=C4=93teris=20Caune?= Date: Mon, 31 Aug 2020 12:32:16 +0300 Subject: [PATCH] Docs: add "Using Runitor" and "Handling More Than 10KB of Logs" sections --- templates/docs/attaching_logs.html | 49 +++++++++++++++++++++++++----- templates/docs/attaching_logs.md | 48 ++++++++++++++++++++++++----- 2 files changed, 81 insertions(+), 16 deletions(-) diff --git a/templates/docs/attaching_logs.html b/templates/docs/attaching_logs.html index 3ed16b33..2617ea2e 100644 --- a/templates/docs/attaching_logs.html +++ b/templates/docs/attaching_logs.html @@ -1,15 +1,15 @@

Attaching Logs

SITE_NAME ping endpoints accept HTTP HEAD, GET and POST request methods.

When using HTTP POST, you can include arbitrary payload in the request body. -If the request body looks like a UTF-8 string, SITE_NAME will log the first 10 kilobytes of -the request body, so you can inspect it later.

+If the request body looks like a UTF-8 string, SITE_NAME will log the +first 10 kilobytes (10 000 bytes) of the request body, so you can inspect it later.

Logging Command Output

In this example, we run certbot renew, capture its output, and submit the captured output to SITE_NAME:

#!/bin/sh
 
 m=$(/usr/bin/certbot renew 2>&1)
-curl -fsS --retry 3 --data-raw "$m" PING_URL
+curl -fsS -m 10 --retry 5 --data-raw "$m" PING_URL
@@ -23,12 +23,45 @@ depending on the exit code:

m=$(/usr/bin/certbot renew 2>&1) if [ $? -ne 0 ]; then url=$url/fail; fi -curl -fsS --retry 3 --data-raw "$m" $url +curl -fsS -m 10 --retry 5 --data-raw "$m" $url -

All in One Line

-

Finally, all of the above can be packaged in a single line. The one-line -version can be put directly in crontab, without using a wrapper script.

+

The above script can be packaged in a single line. The one-line +version sacrifices some readability, but it can be used directly in crontab, +without using a wrapper script:

m=$(/usr/bin/certbot renew 2>&1); curl -fsS --data-raw "$m" "PING_URL$([ $? -ne 0 ] && echo -n /fail)"
-
\ No newline at end of file + + + +

Using Runitor

+

Runitor is a third party utility that runs the +supplied command, captures its output and and reports to SITE_NAME. +It also measures the execution time, and retries HTTP requests on transient errors. +Best of all, the syntax is simple and clean:

+
runitor -uuid your-uuid-here -- /usr/bin/certbot renew
+
+ + +

Handling More Than 10KB of Logs

+

While SITE_NAME can store a small amount of logs in a pinch, it is not specifically +designed for that. If you run into the issue of logs getting cut off, consider +the following options:

+ +
#!/bin/sh
+
+m=$(dmesg | tail --bytes=10000)
+curl -fsS -m 10 --retry 5 --data-raw "$m" PING_URL
+
+ + + \ No newline at end of file diff --git a/templates/docs/attaching_logs.md b/templates/docs/attaching_logs.md index d006a4ae..ee70a931 100644 --- a/templates/docs/attaching_logs.md +++ b/templates/docs/attaching_logs.md @@ -3,8 +3,8 @@ SITE_NAME ping endpoints accept HTTP HEAD, GET and POST request methods. When using HTTP POST, **you can include arbitrary payload in the request body**. -If the request body looks like a UTF-8 string, SITE_NAME will log the first 10 kilobytes of -the request body, so you can inspect it later. +If the request body looks like a UTF-8 string, SITE_NAME will log the +first 10 kilobytes (10 000 bytes) of the request body, so you can inspect it later. ## Logging Command Output @@ -15,7 +15,7 @@ the captured output to SITE_NAME: #!/bin/sh m=$(/usr/bin/certbot renew 2>&1) -curl -fsS --retry 3 --data-raw "$m" PING_URL +curl -fsS -m 10 --retry 5 --data-raw "$m" PING_URL ``` ## In Combination with the `/fail` Endpoint @@ -31,14 +31,46 @@ url=PING_URL m=$(/usr/bin/certbot renew 2>&1) if [ $? -ne 0 ]; then url=$url/fail; fi -curl -fsS --retry 3 --data-raw "$m" $url +curl -fsS -m 10 --retry 5 --data-raw "$m" $url ``` -## All in One Line - -Finally, all of the above can be packaged in a single line. The one-line -version can be put directly in crontab, without using a wrapper script. +The above script can be packaged in a single line. The one-line +version sacrifices some readability, but it can be used directly in crontab, +without using a wrapper script: ```bash m=$(/usr/bin/certbot renew 2>&1); curl -fsS --data-raw "$m" "PING_URL$([ $? -ne 0 ] && echo -n /fail)" ``` + +## Using Runitor + +[Runitor](https://github.com/bdd/runitor) is a third party utility that runs the +supplied command, captures its output and and reports to SITE_NAME. +It also measures the execution time, and retries HTTP requests on transient errors. +Best of all, the syntax is simple and clean: + +```bash +runitor -uuid your-uuid-here -- /usr/bin/certbot renew +``` + +## Handling More Than 10KB of Logs + +While SITE_NAME can store a small amount of logs in a pinch, it is not specifically +designed for that. If you run into the issue of logs getting cut off, consider +the following options: + +* See if the logs can be made less verbose. For example, if you have a batch job +that outputs a line of text per item processed, perhaps it can output a short +summary with the totals instead. +* If the important content is usually at the end, submit the **last 10KB** instead +of the first. Here is an example that submits the last 10KB of `dmesg` output: + +```bash +#!/bin/sh + +m=$(dmesg | tail --bytes=10000) +curl -fsS -m 10 --retry 5 --data-raw "$m" PING_URL +``` + +* Finally, if for your use case it is critical to capture the entire log output, +consider using a dedicated log aggregation service for capturing the logs.