Update packages from crontab - ArchWiki

# Update packages from crontab

Warning: Doing automatic updates from cron is strongly discouraged. It is likely to leave your machine in a broken and unbootable state. If this breaks your machine, do not hold anyone but yourself responsible. You have been warned.

## Do not try this at home!

So, proceed only if you have balls of steel or you were intending to crash your machine anyway. If so, you might as well do it the "right way".

1. First, you (obviously!) need to install cron itself. Do that first.
2. It is highly recommended to also install a mail transfer agent, such as Postfix, to send you notifications if pacman fails.
3. Run as root: `crontab -e`
4. Copy-paste this to your crontab:

    MAILTO=your@email
    LOGFILE=/var/log/cron-pacman.log

    # 1. minute (0-59)
    # |   2. hour (0-23)
    # |   |   3. day of month (1-31)
    # |   |   |   4. month (1-12)
    # |   |   |   |   5. day of week (0-7: 0 or 7 is Sun, or use names)
    # |   |   |   |   |   6. commandline
    # |   |   |   |   |   |
    #min hr  dom mon dow command
    00   13   *   *   *  . /etc/profile && (echo; date; yes |pacman -Syuq) &>>$LOGFILE || (echo 'pacman failed!'; tail $LOGFILE; false)

To check every */time you must use e.g. `*/2` to check every 2 min or any other time value.

If you want to automatically reboot your computer upon a successful upgrade, append '&& reboot' to the above line.

## Do try this at home!

Instead of using `pacman -Syuq` above, use `pacman -Syuwq`. The '-w' will cause pacman to "retrieve all packages from the server, but do not install/upgrade anything." While you will still have to manually update your system, you won't have to wait (as long) for packages to download while doing so.

## Manual set up of auto-update

Note: The sections below are mostly focused on setting up Linux on computer for the home user.
Warning: You must make backup of pacman database before giving computer to user.

If you want to set up Arch Linux for home user and still keep system up to date you will need to make work in background without distraction of user.

Note:

- If you will need to run other scripts from inside of script that run by crone then you will need to use the full path to them, e.g. `pacman -S --needed --noconfirm $(/path/to/script/ShortListUpdates.sh)` or they will not run if you will use only their name without full path.
- Use the `pacman-db-upgrade` command only in the end after installing packages because it may leave the `/var/lib/pacman/db.lck` lock file that will prevent run of the next pacman command. It happens only when scrip is running by cron but not when you start it manually from console.

### Update notifier

There are many programs such as zenity that can show GUI messages from program or scripts that runs in console. In this example you will see a dialog with list with file names and size for each of them that need to be downloaded for installation. By clicking Yes it will return exit code 0 respective 1 for No that will be placed in the $? variable.

    $ zenity --question --text="$(pacman -Qu|awk '//{if(index($0,"[ignored]") == 0 )print $1  }'|pacman -Si -|grep ^[a-Z]| \
     sed 's/ //g'|  awk -F':' '//{ZZ=ZZ+1;XX[ZZ]=$1;SS[ZZ]=$2;}
     END{AA=ZZ/18;for(i=1;i <= AA;i++){print XX[i*18-16]": "SS[18*i-16]" | "XX[i*18-4]": "SS[i*18-4];}}' )"

For a long list of updates is better to use this example and `zenity` can also read data from stdin.

    $ pacman -Qu |awk '//{if(index($0,"[ignored]") == 0 )print $1  }'|pacman -Si -|grep ^[a-Z]|\
      sed 's/ //g'|awk -F':' '//{ZZ=ZZ+1;XX[ZZ]=$1;SS[ZZ]=$2;}
      END{AA=ZZ/18;for(i=1;i <= AA;i++){print XX[i*18-16]": "SS[18*i-16]" | "XX[i*18-4]": "SS[i*18-4];}}' | zenity --text-info --title=Uppdateringar

Is suitable to use after updates has been downloaded.

This will show progress bar. Depends on amount of repositories you will need to calculate correct step size, you can do it simply by dividing 100 with amount of lines printed by pacman or use `LCount=$(grep ^"\[" /etc/pacman.conf -c);echo $((100/LCount))` to calculate and pass it to the awk command.

    $ pacman -Sy --noprogressbar |awk '{AA=AA+16.7;system("echo "AA)}' | zenity --progress --no-cancel

In comparison to zenity the program `xmessage` from xorg-xmessage package shows much smaller windows but doesn't support UTF and has some more limitations but here we will not discuss them, it is just a perfect light notifier.

    ...........
    ...........
    STARTit=$(date '+%M-%S')
    xmessage -timeout 2 "Starting downloading of updates" & disown
    ...........
    ...........
    ENDit="$(date '+%M-%S')"
    xmessage -timeout 3 "${STARTit}"' * '"${ENDit}" & disown
    ...........
    ...........

See also: How to show a message box from a bash script in Linux.

Before running `pacman` you can also add integrity of database check:

    testdb -b "/var/lib/pacman/"
       if [ ${?} != "0" ];then
        zenity --warning --text="Databasfilen är skadat. Kontakta Admin!";
        exit 1;
       fi

### Download updates

The easiest way of creating a short list of updates is by creating the script that will print them to stdout, e.g.

    ShortListUpdates.sh

    #!/bin/bash
    pacman -Qq | \
    grep -e smplayer \
    -e smtube \
    -e ^jre \
    -e ^gst \
    -e firefox | \
    grep -v -e 'to_ignore_package1' -e 'to_ignore_package2' -e 'to_ignore_package3'
    #Files from a group to add
    pacman -Qqg kde | grep -v -e 'to_ignore_package1' -e 'to_ignore_package2' -e 'to_ignore_package3'

Good to download updates after few hours of idle and limit it only to a day time and make similar just that will download and possible also install updates at night time if computer is powered on. To speed up update process at night and lower possible damage add some of packages to ignore list, such as fonts, kernel and mkinitcpio or any other that you think is not at all necessary to be updated. But still you can make a script that user can manually run to update all packages, e.g. `pacman -Qqen | pacman -S --needed --noconfirm -` but do not forget to add safety check of battery status if it is a laptop.

Steps for downloading process

1. Check if stop mark after download exist, e.g. `if [ ! -f /tmp/.downloaded_yes ];then echo Is OK to download;else echo Already downloaded;fi`
2. Check if computer is idle and how long time. Utilities xprintidle[1] for X and command `w` for tty.
3. Check if computer is connected with cable or how much battery is charged. You will need to install upower.

       #!/bin/bash

       MAXpower="60"

       ST=($(upower -i "$(upower -e | grep 'BAT')" | grep -e "state" -e percentage | awk '{print $2}' | sed 's/%//g'))
       if [ ! -z ${ST[0]} ];then
        if [ ${ST[1]} -gt ${MAXpower} ]  || [ ${ST[0]} == 'charging' ] || [ ${ST[0]} == 'fully-charged' ]; then
         echo "OK"
          else
         echo "Fail"
        fi
         else
        echo "OK"
       fi

4. Check type of connection to prevent downloading due huge amount of data if computer using 3G or other PPP connection. Command `ifconfig | grep ^ppp -c`.
5. Check if is online `ping -c 1 8.8.8.8 ; if [ "${?}" != "0" ]; then echo bad; else echo good; fi`
6. Make a short list of the most important updates, e.g. for average internet user will be good to have a web-browser and its plugins,extensions, certifications and SSL componets and some of media players that you configured as default will need to be downloaded.
   Note: Don't make list too big for daily downloads and do not start installation directly after them were downloaded to avoid possible conflicts.
7. Create a stop mark to prevent multiple update checks after download were finished, e.g. `echo downloaded > /tmp/.downloaded_yes`.

Note: The stop mark can be also created after all checks are passed in case if check of idle time is too low and download time might take more time depends on the connection and total size of updates.

### Download and install updates from short list

To download updates from the short list you can use this line `pacman -Sw --noconfirm --needed $(/full/path/to/ShortListUpdates.sh)`.

You will need to add only a power check before continue with installation of updates.

    ...............
    ...............
    yes | pacman -S --noconfirm --needed $(/full/path/to/ShortListUpdates.sh)
    systemctl daemon-reload
    sleep 2
    ...............
    ...............

If you will use a GUI notifier then you may want to use this part in the begin of the update script if updates runs after reboot and user comes to log in screen(Display manager). Be careful with the `sleep` timeout if too low it might freeze the screen. The notifier will be shown even on the login screen that will probably make user wait until updates are finished or after user logged if you will set sleep timeout too high and user logged in before script started.

    #!/bin/bash
    sleep 5
    if [ "$(pgrep X -c)"  != "0" ]; then
    export DISPLAY=:0
    else
    export DISPLAY=:0
    startx "$0"
    fi
    .........
    .........
    .........

You may also will make it remove unnecessary locales for spell check in e.g. `/usr/include/hunspell`, `/usr/lib/aspell-0.60` or any other that contains unnecessary files that created after update or install of packages.

In the crone you can schedule installation of updates from a short list on each boot by using in `crontab`:

    @reboot /usr/local/bin/update

Note: You can also make it copy the update-downloading scripts to `/tmp` to minimize disk usage because the `/tmp` is using RAM for storage of files.

### Download and install at night

If user will leave computer powered on at night then this example could be used to download and install after computer become idle for a some time to be more sure that user will not interrupt.

    ..............
    ..............# Battery and connection safety checks
    ..............
    yes | pacman -S --needed $(pacman -Ssq openjdk | grep -v -e doc -e src)
      if [ $(ifconfig | grep ^ppp -c)  != "0" ];then
        pacman -Suw --noconfirm
       else
        pacman -Qq --native  | pacman -Sw --needed -
      fi
    yes | pacman -Su
    if [ "${?}" != "0"  ];then beep -f 100 -l 1000 && zenity --warning --text="Ett problem har uppstått. Kontakta Admin!"
    if [ -f "/var/lib/pacman/db.lck"  ];then zenity --warning --text="Databasfilen är skadat eller används: /var/lib/pacman/db.lck";fi;
    exit 1
    else
    BkupDate=$(date '+%A-day_of_the_week-%u')
    pacman -Q --native > /opt/.alt_db_bkup-"${BkupDate}"
    date >> ${BkupDate}
    systemctl daemon-reload
    sleep 2
    fi
    pacman-db-upgrade
    ..............
    ..............
    ..............

Some of updates can be installed directly if you want make it download new releases of updates and not only updates for existing. If computer will use wireless or cable connection it will download all even packages that are in the ignore list but install only updates that are not in the ignore list. You can also make it download all updates one day and install next day, for that you will need to modify script and schedule days of the week for downloading and another script for installing or use command line variables to switch from downloading to installing.

### Update packages from the AUR

First must be created the GPG key and added to the `makepkg` configuration file.

    /etc/makepkg.conf

    ...............
    PKGDEST=/path/to/custom_repo
    GPGKEY="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
    ...............

This can be used as a part in the download scripts and in the end of them.

Warning: Sometimes packages can change their names that will prevent update of the equivalent installed or if dependencies will be changed.

    iOK="0"
    URLarray=(PKG1 URL1
    PKG2 URL2)

    arrP=0;arrU=1

    while [ "${#URLarray[@]}" -gt "$arrU"  ];do

     wget -nd -nH -m "${URLarray[arrU]}" -P /tmp
      if [ "$?" != "0"   ];then
       pacman -Sw "${URLarray[arrP]}" --needed --noconfirm
    else
    cd /tmp
    runuser -u user_name makepkg
      if [ "$?" == 0 ];then iOK="1";fi
    rm /tmp/PKGBUILD
    fi
     arrU=$((arrU+2))
     arrP=$((arrP+2))
    done

    if [ "$iOK" != "0" ]; then
    repo-add -s -n /path/to/custom.db.tar  /path/to/*.xz
    fi

An alternative for a more complex downloads when some of the files need to be downloaded from other sources like e.g. github. This should be repeated for each file.

    wget -nd -nH -m https://aur.archlinux.org/packages/....../PKGBUILD -P /tmp
    wget -nd -nH -m  https://raw.githubusercontent.com/....../filename -P /tmp
    cd /tmp
    runuser -u babak makepkg
    if [ "$?" == 0  ]; then
    repo-add -s -n /opt/.AUR/x86_64/custom.db.tar /opt/.AUR/x86_64/*.tar.xz
    fi
    rm /tmp/PKGBUILD
    rm /tmp/filename

Note: An alternative to update packages is to download and unzip binaries if are available for your architecture. But the version of packages will not be registered in the pacman database. The best way if it is possible then download by using a single URL instead of parsing the downloaded file for the URL.

Example how to download and install pepper flash binaries. The script is using p7zip for extraction of the rpm to stdout.

    #......... #Check if the destination folders are exist and notifiers if needed and if running as root
    #......... #
     wget -m -nd -nH https://dl.google.com/linux/direct/google-chrome-stable_current_"$(uname -m)".rpm -P /path/to/.AUR/chromium-pepper-flash -o /tmp/chromium-pepper-flash.wget-log
       if [ "$(grep '100%' -c /tmp/chromium-pepper-flash.wget-log)" != "0" ];then
       if [ ! -d "/tmp/PepperFlash/" ];then mkdir /tmp/PepperFlash/ --mode=755 -p ;fi
        cd "/tmp/PepperFlash/"
        7z e /path/to/.AUR/chromium-pepper-flash/google-chrome-stable_current_"$(uname -m)".rpm -so | cpio -idvm
       chmod 644 /tmp/PepperFlash/opt/google/chrome/PepperFlash/*
       cp --preserve=all -v /tmp/PepperFlash/opt/google/chrome/PepperFlash/* /usr/lib/PepperFlash/
       rm -fr /tmp/PepperFlash/.
    #......... #
    #......... # Use notifier if needed
      fi

### Update the mirrorlist file

This can be scheduled to update mirrors once a week or few times in a month only for countries you want

    rm  /etc/pacman.d/mirrorlist
    for Cnt in China Iran Russia Korea;
     do
    awk -v GG=$Cnt '{if(index($0,GG) != 0)AA=1;if(AA == 1)
       {if( match($0,"#") != "0"){SS=$0;sub("#","",SS);print SS ;}else AA=0} }' \
     /etc/pacman.d/mirrorlist.pacnew >> /etc/pacman.d/mirrorlist;

    done

See also: What to do if pacman-mirrorlist is not installed.

## Recovery after failed update or upgrade

If user did upgrade or update that prevents boot into the desktop environment or applications cannot start but user is able to see the display manager then it is good to make a separate environment for the user that is easy to use to try fix those errors on its own. This will be useful for users that has relatively low knowledge about computer specially about Linux. Install openbox or any other equivalent window manager that they can use with the idesk and create own buttons to the programs or better scripts that are configured to make actions and show warnings and the administrator contact information before starting. To prevent any unwanted user to use the "failsafe" environment or some of the recovery tasks you can add to begin of the script or window manager autostart something like:

    AllowedUser="JolinTsai";
    if [ "$(whomai)" != " ${AllowedUser,,}" ];then zenity --warning --text="You are not the allowed user!
    The allowed user is "$AllowedUser;
    #openbox --exit
    exit 1
    fi

You will need to remove all unnecessary entries from "type of session" list, you can do it by removing or better to moving the *.desktop files which contains information about them to the backup folder and create your own with a custom configuration, to make it simply just copy one of the *.desktop files to a file with the name you want and make changes in the Exec and description parts.

Something like this you can use to remove or move files to backup directory.

    find /usr/share/apps/kdm/sessions/ -maxdepth 1 -type f \
    ! -name kde-plasma.desktop ! -name lxde.desktop ! -name openbox.desktop ! -name xfce4.desktop \
    -exec mv "{}" /usr/share/apps/kdm/sessions/OLD_/ \;

You usually can find them in some of those folders:

    /usr/share/config/kdm/sessions
    /usr/share/apps/kdm/sessions
    /usr/share/xsessions/

You will also need to remove borders from windows to prevent user to close the working window such as xterm if you will use it to show output of commands while they are working. You can do it by using the devilspie.

To get list of the window names for using in the devilspie configuration file you can use the wmctrl utility `wmctrl -l | awk '{print substr($0, index($0,$4))}'` or when you start devilspie you will see all information that is possible to use in the configuration file.

`Window Title: 'name@host:~/.path'; Application Name: 'name@host:~/.path' ; Class: 'XTerm' ; Geometry: 492x350+487+226`

The `window_name` , `application_name` and `window_class` can be used to change the window properties.

Example of the devilspie configuration file that you can use for the preferred application

    .devilspie/DesktopConsole.ds

    (if (is (window_class) "XTerm")
            (begin
    (undecorate)
    (skip_tasklist)
    (above)
    (fullscreen)
    (maximize)
    (unpin)
    (skip_pager)
                )
            )

The XTerm has also a command line to start it in the full screen: `xterm -fullscreen`.

Tip: To run an application in the full screen makes user unable to click on other buttons on the desktop until task is completed.

### Download and fix damaged packages

The factual accuracy of this article or section is disputed.

Reason: The bash script has mistakes in its code. (Discuss)

If some of the applications got broken then it much easier to download it from the website and extract. By reading logs is possible to determinate which of applications is broken.

Example how to download the pacman package or install it if available in the pacman cache.

    #!/bin/bash
    RepoType=(core extra community)
    Arch="x86_64"
    PacCache="/var/cache/pacman/pkg"
    LogName="down_pkg_log_"
    PathPKG="/tmp"

    PacName="pacman"
    RepoSelect="0"
    #PacName="$1"
    #RepoSelect="$2"

    PacFind="$(ls -1 ${PacCache} --sort=version|awk -v QQ=$Arch -v EE=$PacName '//{if( match($0,EE"-[0-9]") == "1" )ZZ=$1 }
    END{if(index(ZZ,QQ) != 0 )print ZZ}')";

    if [ ! -z "$PacFind" ];then
     tar --exclude-ignore='.MTREE' --exclude-ignore='.PKGINFO'  --exclude-ignore='.INSTALL' -C / -xvf "${PacCache}/${PacFind}"
    else
     wget --content-disposition -m -nd -nH https://www.archlinux.org/packages/${RepoType[RepoSelect]}/${Arch}/${PacName}/download/ -P ${PathPKG} -o "${PathPKG}/${LogName}"
      if [  "$?" != "0" ];then
     xmessage "No Internet connection is available
    and
    cannot find the $PacName file in the pacman cache."
    exit 1
      else
    UnTarPKG="$(awk -v XX=${PacName} -F"$PathPKG" '//{AA=$1;if( $2 ~ "/" && index($2,"[") == "0"  ){KK=$2;
    sub(".","",KK);split(KK,SS,"xz");print SS[1]"xz" }}' /tmp/down_pkg_log_)"
     tar --exclude-ignore='.MTREE' --exclude-ignore='.PKGINFO'  --exclude-ignore='.INSTALL' -C / -xvf "${PathPKG}/${UnTarPKG}"
      fi
    fi

See also: What to do if pacman is broken beyond repair

Note:

- For some programs is enough to remove their configuration files and they will create new on the next start but other need to be restored from a back to preserve made by you configuration for the user.
- To prevent user to delete configuration files by accident you can lock folders from browsing with the `chmod 300 .` or use 733 mask if you will change owner to root or any other admin user that has permission to use sudo command and do `chmod 444 file_name` for the files like .bash_logout , .bash_profile , .bashrc , .xinitrc or any other in the user home folder that has no need to be modified or removed.

Category:

- Package management