Thursday, 9 March 2017

Introducing mailing in crontab-ui

Now crontab-ui has option to send mails after execution of jobs along with output and errors attached as text files. This internally uses nodemailer and all the options available through nodemailer are available here.

Defaults


To change the default transporter and mail config you can modify config/mailconfig.js.
var transporterStr = 'smtps://user%40gmail.com:password@smtp.gmail.com';

var mailOptions = {
    from: '"Fred Foo 👥" <foo@blurdybloop.com>', // sender address
    to: 'bar@blurdybloop.com, baz@blurdybloop.com', // list of receivers
    subject: 'Job Test#21 Executed ✔', // Subject line
    text: 'Test#21 results attached 🐴', // plaintext body
    html: '<b>Test#21 🐴</b> results attached' // html body
};

Troubleshooting


Make sure that you have node at /usr/local/bin/node else you need to create a softlink like this
ln -s [location of node] /usr/local/bin/node

Setting up crontab-ui on raspberry pi

In this tutorial I will show you how to setup crontab-ui on raspberry pi.

Step 1

Find your architecture
uname -a
Linux raspberrypi 4.4.50-v7+ #970 SMP Mon Feb 20 19:18:29 GMT 2017 armv7l GNU/Linux
Note that it is ARMv7. Download and extract latest node.
wget https://nodejs.org/dist/v7.7.2/node-v7.7.2-linux-armv7l.tar.xz
tar xz node-v7.7.2-linux-armv7l.tar.xz
sudo mv node-v7.7.2-linux-armv7l /opt/node

Step 2

Remove old nodejs if it is already installed and add the latest node to the $PATH
sudo apt-get purge nodejs
echo 'export PATH=$PATH:/opt/node/bin' > ~/.bashrc
source ~/.bashrc

Step 3

Install crontab-ui and pm2. And start crontab-ui.
npm install -g crontab-ui
npm install -g pm2
pm2 start crontab-ui
Now your crontab-ui must be running. Visit http://localhost:8000 on your browser to see if it is working.

Step 4 (Optional)

In order to be able access crontab-ui from outside, you have to forward the port 8000. Install nginx and configure.
sudo apt-get install nginx
sudo vi /etc/nginx/sites-available/default
Paste the following lines in the file:
server {
    listen 8001;

    server_name localhost;

    location / {
        proxy_pass http://localhost:8000;
    }
}
Restart nginx
sudo service nginx restart
Now, crontab-ui must be accessible from outside through port 8001. So, to access crontab-ui, go to
<ip address of pi>:8001
You can also setup http authentication by following this.
Thanks!
Fork me on Github

Saturday, 28 January 2017

My solutions to cmdchallenge

I recently stumbled upon https://cmdchallenge.com which sort of tests your command line knowledge and comfortability. You have to basically solve all the challenges in a single line of bash. It is pretty simple and fun. You should give it a try before checking the solutions.


hello_world/

# Print "hello world".
# Hint: There are many ways to print text on
# the command line, one way is with the 'echo'
# command.
# 
# Try it below and good luck!
# 
Solution:
echo "hello world"

current_working_directory/

# Print the current working directory.
#
Solution:
pwd

list_files/

# List all of the files in the current
# directory, one file per line.
#
Solution:
ls -1

last_lines/

# Print the last 5 lines of "access.log".
# 
Solution:
tail -5 access.log

find_string_in_a_file/

# There is a file named "access.log" in the
# current working directory. Print all lines
# in this file that contains the string "GET".
#
Solution:
grep GET access.log

search_for_files_containing_string/

# Print all files, one per line that contain
# the string "500".
# 
Solution:
grep -rl * -e 500

search_for_files_by_extension/

# Print the relative file paths, one path
# per line for all files that start with
# "access.log" in the current directory.
# 
Solution:
find . -name "access.log*"

search_for_string_in_files_recursive/

# Print all matching lines (without the filename
# or the file path) in all files under the current
# directory that start with "access.log" that
# contain the string "500".
# 
Solution:
find . -name "access.log*" | xargs grep -h 500

extract_ip_addresses/

# Extract all IP addreses from files that
# that start with "access.log" printing one
# IP address per line.
# 
Solution:
find . -name "access.log*" | xargs grep -Eo '^[^ ]+'

delete_files/

# Delete all of the files in this challenge
# directory including all subdirectories and
# their contents.
# 
Solution:
find . -delete

count_files/

# Count the number of files in the current
# working directory. Print the number of
# files as a single integer.
# 
Solution:
ls | wc -l

simple_sort/

# Print the contents of access.log
# sorted.
# 
Solution:
sort access.log

count_string_in_line/

# Print the number of lines
# in access.log that contain the string
# "GET".
# 
Solution:
grep GET access.log | wc -l

split_on_a_char/

# The file split-me.txt contains a list of
# numbers separated by a ';' character.
# Split the numbers on the ';' character,
# one number per line.
# 
Solution:
cat split-me.txt | sed s/\;/\\n/g

print_number_sequence/

# Print the numbers 1 to 100 separated
# by spaces.
# 
Solution:
echo {1..100}

remove_files_with_extension/

# There are files in this challenge with
# different file extensions.
# Remove all files with the .doc extension
# recursively in the current working directory.
#
Solution:
find . -name "*.doc" -delete

replace_text_in_files/

# This challenge has text files that contain
# the phrase "challenges are difficult". Delete
# this phrase recursively from all text files.
# 
Solution:
find . -name "*.txt" -exec sed -i 's/challenges are difficult//g' {} +

sum_all_numbers/

# The file sum-me.txt have a list of numbers,
# one per line. Print the sum of these numbers.
#
Solution:
cat sum-me.txt | xargs | sed -e 's/\ /+/g' | bc

just_the_files/

# Print all files in the current directory
# recursively without the leading directory path.
# 
Solution:
find . -type f -printf "%f\n"

remove_extensions_from_files/

# Remove the extension from all files in
# the current directory recursively.
# 
Solution: (note you cant use find .)
find `pwd` -type f -exec bash -c 'mv "$1" "${1%.*}"' - '{}' \;

replace_spaces_in_filenames/

# The files in this challenge contain spaces.
# List all of the files in the current
# directory but replace all spaces with a '.'
# character.
# 
Solution:
find . -type f -printf "%f\n" | xargs -0 -I {} echo {} | tr ' ' '.'

files_starting_with_a_number/

# There are a mix of files in this directory
# that start with letters and numbers. Print
# the filenames (just the filenames) of all
# files that start with a number recursively
# in the current directory.
# 
Solution:
find . -name '[0-9]*' -type f -printf "%f\n"

print_nth_line/

# Print the 25th line of the file faces.txt
# 
Solution:
sed '25q;d' faces.txt

remove_duplicate_lines/

# Print the file faces.txt, but only print the first instance of each
# duplicate line, even if the duplicates don't appear next to each other.
# 
Solution:
awk '!seen[$0]++' faces.txt

corrupted_text/

# You have a new challenge!
# The following excerpt from War and Peace is saved to
# the file 'war_and_peace.txt':
# 
# She is betraying us! Russia alone must save Europe.
# Our gracious sovereign recognizes his high vocation
# and will be true to it. That is the one thing I have
# faith in! Our good and wonderful sovereign has to
# perform the noblest role on earth, and he is so virtuous
# and noble that God will not forsake him. He will fulfill
# his vocation and crush the hydra of revolution, which
# has become more terrible than ever in the person of this
# murderer and villain!
# 
# The file however has been corrupted, there are random '!'
# marks inserted throughout.  Print the original text.
# 
Solution: (Found this on hackernews)
< war_and_peace.txt tr -s '!' | sed 's/!\([a-z]\)/\1/g' | sed 's/!\( [a-z]\)/\1/g' | sed 's/!\.!/./g' | sed 's/ !/ /g'


Also, you can checkout the creator's solutions here.

Thursday, 19 January 2017

Look before you paste from a website to terminal

Most of the time when we see a code snippet online to do something, we often blindly copy paste it to the terminal. Even the tech savy ones just see it on the website before copy pasting. Here is why you shouldn't do this. Try pasting the following line to your terminal (SFW)

ls ; clear; echo 'Haha! You gave me access to your computer with sudo!'; echo -ne 'h4cking ## (10%)\r'; sleep 0.3; echo -ne 'h4cking ### (20%)\r'; sleep 0.3; echo -ne 'h4cking ##### (33%)\r'; sleep 0.3; echo -ne 'h4cking ####### (40%)\r'; sleep 0.3; echo -ne 'h4cking ########## (50%)\r'; sleep 0.3; echo -ne 'h4cking ############# (66%)\r'; sleep 0.3; echo -ne 'h4cking ##################### (99%)\r'; sleep 0.3; echo -ne 'h4cking ####################### (100%)\r'; echo -ne '\n'; echo 'Hacking complete.'; echo 'Use GUI interface using visual basic to track my IP'
ls
-lat


It should look something like this once it is pasted onto your terminal.
View post on imgur.com

You probably guessed it. There is some malicious code between ls and -lat that is hidden from the user

Malicious code's color is set to that of the background, it's font size is set to 0, it is moved away from rest of the code and it is made un-selectable (that blue color thing doesn't reveal it); to make sure that it works in all possible OSes, browsers and screen sizes.




This can be worse. If the code snippet had a command with sudo for instance, the malicious code will have sudo access too. Or, it can silently install a keylogger on your machine; possibilities are endless. So, the lesson here is, make sure that you paste code snippets from untrusted sources onto a text editor before executing it.

Thanks for reading!

Tuesday, 18 October 2016

How to download large folders on dropbox

Recently, someone shared a large folder with me and when I tried to download it, I was getting an error; "There was an error downloading your file".
This error seemed very vague and after a quick search online, I figured out that it is not possible to download folders which are bigger than 1 GB.  And according to dropbox's help article, I will be able to download it only if I add it to my dropbox. With dropbox's puny 2GB free storage it was not possible and I was not ready to spend $$$ just for this. 
So, I wrote a simple script in javascript that I can run it on browser console to download all files in a folder!

How to do it?


Step 1. Navigate to the dropbox folder on the browser and open your developer console. Press   cmd + j  on mac or  ctrl + shift + j  on linux and windows.

Step 2. Paste the following code in the console.



Step 3. The browser will try to block the windows trying to download it. Select the option to  Always allow pop-ups from https://www.dropbox.com . For instance this is how it will look on Google Chrome (you have to click on right most icon in the search bar).


Step 4. Your files will be downloaded one by one!

NOTE: If any folder inside the folder is greater than 1GB in size, then you may have to do the same process after navigating to that folder in the browser.

Thursday, 15 September 2016

Right way to set env variable while exec or execFile in nodejs

According to the official documentation, exec allows you to pass additional environment variables as part of options like this:

This looks fine right? except that it totally isn't! By passing "env" as an option, you are not adding on to existing environment variables, but you are replacing it.  This is not clear from the documentation and can leave you scratching your head for a while as it can seem to break the command for no particular reason at all! So, you need to essentially make a copy of process.env and modify it like follows.


Thanks for stopping by!

Thursday, 28 April 2016

Problem with clipboard on Ubuntu

Clipboard on Ubuntu is "broken". Well, maybe not. But copy-paste is broken in a lot of applications on Ubuntu. Let me give you an example,

  • Open LibreOffice.
  • Write something and copy it.
  • Paste it somewhere else. It works as expected.
  • Now close LibreOffice and try pasting. It won't work.
This is a well known bug - Copy-Paste doesn't work if the source is closed before the paste; affecting a lot of applications on Ubuntu. And they are not so keen on fixing it, atleast not any time soon.
The reason for this is that these applications do not comply with the clipboard specification from FreeDesktop
If a client needs to exit while owning the CLIPBOARD selection, it should request the clipboard manager to take over the ownership of the clipboard, using the SAVE_TARGETS mechanism. If there is no clipboard manager, or if the SAVE_TARGETS conversion fails, the application should simply exit.
Applications need to transfer ownership of the clipboard to clipboard manager before exiting for copied data to perisist after it exits. There is certainly nothing you can do about it, unless offcourse you are willing to modify the source code of each of these applications to make it comply to FreeDesktop specs.

Fix. Well.. workaround.

The reporter/ moderator of the bug report suggests that we should install diodon, klipper, glipper, parcellite or xfce4-clipman as a workaround for this issue.
Working of diodon


Tuesday, 26 April 2016

Ubuntu 16.04 won't wake up from suspend

I recently installed Ubuntu 16.04 LTS Xenial Xerus on my Thinkpad E550. I honestly regretted it not just because it doesn't support AMD proprietary fglrx driver aka AMD Catalyst or AMD Radeon Software but because the suspend feature stopped working.

I initially thought that this had something to do with video drivers that I had installed on Ubuntu 15.10 which were now incompatible with 16.04. I realized that this was not the case as the issue persisted even on opensource drivers that it is compatible with.

Also on a closer observation I realized that it was not that my system was not able to wake up from suspend, but that it was not able to suspend at all. On suspending, the screen would go off but my laptop kept running, heating up and draining battery. This problem existed when I hibernate or shutdown as well.

Now the only possible reason for this is some problem in acpi which is not letting my system to suspend. Ubuntu 16.04 is shipped with kernel 4.4. A quick search on this issue on kernel 4.4 made me realize that this exists across several destros and mostly on thinkpads. So I upgraded to kernel 4.5 and the problem is resolved.

Installing kernel 4.5

32 bit

64bit

Then reboot!

Friday, 8 January 2016

Parsing wav file in node.js

I have worked with wav audio data in python before. Scipy provides a very nice way to do this using scipy.io.wavfile


I wanted to do exactly the same in node.js. There is a module called wav which sort of does it. However I faced several problems.

1. The Reader() method reads the file stream and converts into chunks of Buffer. This is very good while building web apps as you can send chunks of data separately and combine it later. However, I was just writing a script that'd run offline, So I had to put all the buffers to an array and then use the concat method of Buffer.


2. The wav module doesn't do much processing and just throws the raw binary information at us. So, we have to take care of

  • Combining hex data to get amplitudes - One frame can be represented using several blocks of 8-bit hex. The number of blocks per frame is got using blockAlign parameter in the format.
  • Endianness - Data may or may not be  in little endian format. So, while reading the blocks of hex data, we have to take care of this.
  • Handling negative amplitudes - Frames spanning several blocks when negative can be little challenging to handle as they are just stored as their two's complement.
  • Separating channels - The raw binary contains data of all the channels together. Fortunately, they are mentioned one after the other. Using channels parameter in the format, channels can be separated easily.
So, let's see how I handled all the above cases. First, we need to capture the "format" of the audio file that contains information about the file.


Then, on "end" event i.e after all the chunks have been combined, we will handle all the cases mentioned above as follows


Concluding, in this article I showed you how to handle and parse wav files in node.js by resolving several problems such as merging chunks of Buffer, combining hex data to get amplitudes, endianness, handling negative amplitudes and separating channels.

Wednesday, 23 December 2015

Unfortunately the process com.android.phone has stopped in CM 13

I have a good old nexus 4 still trying to keep up with the big boys in the market. After google ditched the plans of releasing the latest android 6.0 - Marshmallow for nexus 4, things didn't look so great. But with the release of CM 13 (mako), even nexus 4 has android 6.0! Previously, I was running CM 12.1 and I dirty flashed CM 13 on top of it. Although it doesn't have all the features that CM 12 had, with time, the awesome guys in XDA and cyanogen will port all the features to CM 13.

After dirty flashing CM 13 on top of CM 12.1, I faced some problems. First of all, things were kinda slow. So I went to Developer options in Settings and set all the Animation scaling to .5x. This considerable made things faster. Secondly and more importantly, my sim card was not getting detected and I was getting a ton loads of popups saying "Unfortunately the process com.android.phone has stopped".


I looked at the logcat to see what is going wrong and I found that this was appearing a lot of times.


Clearly, the problem was with com.android.providers.telephony. Upon looking closely, the problem was related to CursorWindow and Cursor. Hence, the problem was not with CM 13 but with the old database left behind by CM 12.1. 

Solution


Just delete the folders /data/data/com.android.providers.telephony and /data/data/com.android.phone using some tool like Root Explorer or you can do this in adb shell.

rm -rf /data/data/com.android.providers.telephony
rm -rf /data/data/com.android.phone

Then, restart the phone.