The ed editor

It might happen as system administrator to have to work, especially to troubleshoot something, on an old Linux distro or Unix system where you don’t have none of the following known editors: vi, Emacs, Joe, ex, Sam or sed. In this case you will have at least ed, one of the oldest Unix inline text editor. So, you have guessed, this article will focus on this less famous, by today standard, editor. We will illustrate examples not just extracts from the man pages, which is not too much useful for novices.

The ed editor, like vi, works basically in two modes, edit and command. Once you start ed you will be presented with this screen without any prompt and in command mode:

Nothing is happening yet, you need to display the prompt by pressing P followed by Enter, as you can see an asterisk will be shown:

Afterwards you can type the letter i followed by newline (insert), to start typing something:

ed

Note that you must “announce” you want to finish editing with a . in a line on its own.
The number 13 is the number of characters written to the file.
You can save and exit ed with the command w (and the name of the file) and q, like in vi.

ed

To open an existing file we just runs ed followed by the filename.
Because is a line editor we won’t see the all content of the file by default, but just how many characters we have.
Let’s say we want to see its content. In this case we simply type p and press enter:

ed

We want to add the same word but in the different language, so we switch into “insert” mode
So as before we switch into command mode and with i we edit and add the line below, then we save without specifying
the filename as we already did by opening this file:

ed

Now we want to display the whole content of the file, we can do that with the command ,p
We can also this display the last line by simply using p

ed

As you can see the last line we typed is, by mistake, on top and not in the bottom.
There is also a small typo : a missing exclamation mark; let’s fix this typo first.
We just move to the desired line by typing its number followed by Enter
Then to amend the line we will use a sed like syntax.


OK we have selected the line and now we type command
s/$/!/p
to add that specific character at the end of the line ($ means EOL).

ed

Now it’s time to move our line to the bottom: you basically specify with the command m the desired line number. We must know the total number of lines as we cannot enter a not existing line.

ed

Now we want to add a new line or better to append it, we will use the command a for that.

ed

Now, let’s say we want to remove a line, we will use d for that.

ed

Finally, let’s say we want to append the result of a command, we can use the command !r for that:

ed

Note that the ! alone will just have the aim to run some system command without exiting ed, like in vi.ed

This is it: I hope you found interesting even if you won’t be using it often.

Security news

A potential bypass of the sudo runas security mechanisms has been discovered; When sudo is configured to allow a user to run commands as an arbitrary user via the ALL keyword in a Runas specification, it is possible to run commands as root by specifying the user ID -1 or 4294967295.

For more info, please consult the sudo website at https://www.sudo.ws/alerts/minus_1_uid.html

IT pill

Classic shell scripting (Bash, Korn Shell, etc.) is like démodé nowdays, but neverthless still used in Linux and other Unix like systems.
Therefore is still worth to mention about a static analysis tool written in Haskell https://www.shellcheck.net/
This tool will analyze your script and display each case of improper code or not following best practices.

Access Control Lists – part 1 –

Access Control List (ACL) in Linux provides an additional level of security. Traditionally we have the owner, the group and the other users permissions on a given resource; but what about giving the same owner permissions to an additional user?
Normally this is done by adding the user to the group set on that file or folder or giving more permissions to all other users; of course the second choice is not recommended for security reasons.

ACL answers precisely this need, giving additional privileges to specific users, without adding them to the same owner group which would grant also privileges to other resources. And again without giving more permissions to other.

So an Access Control List allows a system administrator a more granular control over users permissions than the classic ALL or NOTHING approach.

Some usage examples:

1) We want an application to write to a user subfolder, but without adding the application user id to the user group or setting other permissions.

2) A team needs to copy some data to a root home subfolder and we don`t want to add them to the root group or worse give them the root password.

3) There are several users belonging to a group and work on a common folder; however we want a specific subfolder in it to be writable and executable only by a specific user, not by all users in that group.

4) We have a database folder containing personal data and here we want to grant read and executable permissions to someone in the application team, but revoke all permissions for other . Also we want to decouple the application functional ID privileges from the user privileges; that means we won`t add the user to the same group as the application.
Then we will add a further ACL for a user in the application team with full permissions to the same resources.

In the above examples we can also make use of groups, as with ACL we can grant additional groups permissions as well.

Transferring Data with BBCP

Rsync is a quite known utility if we want synchronize data between two nodes or even between two folders in the same system, and of course also used for backup purposes. Its main peculiarity is that it sends only incremental changes; for instance, if you get a connection drop when using classic FTP you typically have to re-send the entire folder or file, but with this tool you re-run it again and it will start from where it left.

However Rsync is not so good if you want speed and it doesn`t have any multithreaded capability; so if you have GBs or even 1 TB of data to transfer it can be quite slow in the end.
To overcome these drawbacks you can use BBCP, which is a point-to-point network utility written by Andy Hanushevsky at  SLAC with the goal to get close to the line speed in a LAN or WAN connection.

You don’t need any server listening or any SSH daemon, but you have to make sure bbcp is installed and in the PATH in BOTH systems.

Assuming GIT is installed, below are the steps to install it:

$ git clone http://www.slac.stanford.edu/~abh/bbcp/bbcp.git

 

$ cd bbcp/src
Linux
$ make

 
The pre-requisites are:

GNU C++ compiler
Zlib library
Pthreads library

 

Some examples of usage are:

bbcp -P 2 -V -w 8m -s 16 /local/path/bigfile.tar remotesystem:/remote/path/bigfile.tar

-V verbose output
-P 2 display progress every two seconds
-s 16 create 16 parallel network streams (or threads)
-w sets to 8 MB the size of the disk input/output (I/O) buffers

 

To transfer a directory just use the -r option (it stands for recursive of course).

bbcp -r -P 2 -V -w 8m -s 16 /local/path/* remotesystem:/remote/path

To resume files in case of a lost connection add the -a and -k switch.

bbcp -r -k -a -P 2 -V -w 8m -s 16 /local/path/* remotesystem:/remote/path

If a firewall is blocking the communication between source and destination, use the -z option

bbcp -P 2 -V -w 8m -a -k -z -s 16 /local/path/bigfile.tar remotesystem:/remote/path/bigfile.tar

 

I have tested BBCP only on Linux, but it should be possible to compile it and install it in all major UNIX systems. The Windows platform  is not supported.

Resources:
http://www.slac.stanford.edu/~abh/bbcp/
http://pcbunn.cithep.caltech.edu/bbcp/using_bbcp.htm
https://www.olcf.ornl.gov/kb_articles/transferring-data-with-bbcp/