Move past transactions from one journal to another and also automatically generate and import periodic transactions like subscriptions

Wanted to share this simple bash script.

I find it highly useful for my workflow.

I tried to completely describe how it works and what it does with comments inside the script.

#!/bin/bash

## PLS HAVE PROPER BACKUPS BEFORE RUNNING THIS SCRIPT. IT MIGHT DELETE ALL YOUR JOURNALS!!!!

## Name of this script: "~/.local/bin/hledger-due-transactions.sh"

#------------
## This is a script that I always run when opening hledger-ui. Example alias:

### hledger check -s -f ~/hledger/config.journal && hledger-due-transactions.sh && hledger-ui -f ~/hledger/config.journal --pretty=yes -e=2months -E  --theme=greenterm --color=yes -f ~/hledger/forecast.journal

## It enables you to automatically add periodic transactions like subscriptions, which you then only have to confirm (*) and optionally edit/correct...
## It also is able to "move" past transactions from one journal to another. Useful if you have separate journals for certain future transactions or you use https://github.com/tazzben/DepreciateForLedger...
#-------------

#-------------
## My main Journal for the current year. Only contains transactions, configuration is only done in config.journal (which then includes other journal files like: "include main.journal")
## This script adds the generated and other past transactions to this file!!!
MAIN_JOURNAL="$HOME/hledger/main.journal"
#-------------

## Generated from a periodic.journal just for view in hledger-ui
LEDGER_RECURRING="$HOME/hledger/forecast.journal"
#-------------

#-------------
## Generated from the periodic.journal just to import future transactions
IMPORT_LEDGER_RECURRING="$HOME/hledger/imports/forecast.journal"
#-------------

#-------------
## File to manually add periodic transactions like:

### ~ every 20th day of month  Proton Unlimited
###    assets:bank
###    expenses:proton-unlimited  $11.99

## This script will generate these transactions for a set period of time and import every transactions that is newer than the one specified
## in the in the .latest."$JOURNAL-NAME".journal in the same dir as the journal. Example off such a file with 2 periodic transactions (.latest.forecast.journal) :

### 2024-10-10
### 2024-10-10

## for more info look at: https://hledger.org/1.40/hledger.html#import
PERIODIC_JOURNAL="$HOME/hledger/periodic.journal"
#-------------

#-------------
## Extra Journal to manually only add product preorders ...
PREORDER_JOURNAL="$HOME/hledger/preorder.journal"
## Do the import optionally in a different dir...
#IMPORT_PREORDER_JOURNAL="$HOME/hledger/imports/preorder.journal"
#-------------

#-------------
## Journal to manually record future transactions
FUTURE_JOURNAL="$HOME/hledger/future.journal"
## Do the import optionally in a different dir...
#IMPORT_FUTURE_JOURNAL="$HOME/hledger/imports/future.journal"
#-------------

#-------------
## OPTIONAL add a log file to the script... (append every single import run...)
#IMPORT_LOG_FILE="$HOME/hledger/imports/import.log"
## Add newline to the log file
#echo -e "\n" >> "$IMPORT_LOG_FILE"
## Add timestamp to the log file
#date >> "$IMPORT_LOG_FILE"
#-------------

#-------------
## Variable that defines when the generated transactions end/begin
IMPORT_DATE=1day
#-------------

#-------------
## create a separate forecasting.journal for view in hledger-ui... (this creates a .journal starting one day into the future)
## This enables you to see how you future balance will most likely look like.
hledger print -f "$PERIODIC_JOURNAL" --forecast --begin=1day --end=10year -o "$LEDGER_RECURRING"
#-------------

#-------------
# this function creates a .journal that ends today...
create-forecast_function () {
hledger print -f "$1" --forecast --begin=-1year --end=1day -o "$2"
}

## run the function to generate the periodic transactions for import
create-forecast_function "$PERIODIC_JOURNAL" "$IMPORT_LEDGER_RECURRING"
## Example output:

### no new transactions found in /home/user/hledger/imports/forecast.journal
#-------------

#-------------
# this imports recurring transaction up to 1 year in the past( defined in the "create-forcast_function" ) to the the main.journal
# (but only transactions that are newer than the date written in .latest.$JOURNAL.journal see hledger docs for more infos)
# WHICH IS PROPABLY WHY YOU SHOULD CREATE THE .latest.$JOURNAL.journal YOURSELF IF IT IS YOUR FIRST TIME RUNNING THIS SCRIPT
importtransactions_function () {
  ## Optionally log the import process
  #hledger import -f "$MAIN_JOURNAL" "$1" >> "$IMPORT_LOG_FILE" 2>&1
  hledger import -f "$MAIN_JOURNAL" "$1"
  echo -e "\n\n"
}

importtransactions_function "$IMPORT_LEDGER_RECURRING"

#-------------

#-------------
## This function will generate a new journal in a temp file with all past transactions as defined in "$IMPORT_DATE" and then import this file to the "$MAIN_JOURNAL"
## Then it will generate a new journal in the same temp file with all transactions that are not in the past as defined in "$IMPORT_DATE" and use this file to override the original file.
## Then it will remove the tempfile. Note that if this script gets interrupted it probably wont be deleted...
movetransactions_function () {
tmpfile=$(mktemp) &&\
  hledger print -e="$IMPORT_DATE" -f "$1" > "$tmpfile" &&\
  hledger import -f "$MAIN_JOURNAL" "$tmpfile" &&\

  ## Add comments to the output so you can see which file gets currently imported. Otherwise you will only see:
  ###  no new transactions found in /tmp/tmp.ptXvZxxGTw
  ## instead of
  ### no new transactions found in /tmp/tmp.ptXvZxxGTw
  ### Import from: /home/user/hedger/future.journal
  echo "Import from: $1" &&\
  echo -e "\n\n" &&\

  ## Print new file (the one from which you want to import from without the imported transactions)
  ## THIS WILL OVERRIDE THE ORIGINAL FILE
  hledger print -b="$IMPORT_DATE" -f "$1" -o "$1" ;\
  ## Copy the file instead if 'hledger print -b="$IMPORT_DATE" -f "$1" -o "$1"' doesn't work properly and instead erases your file (works fine on Fedora Workstation)
  #  hledger print -b="$IMPORT_DATE" -f "$1" > "$tmpfile" &&\
  #  cp "$tmpfile" "$1" ;\

rm "$tmpfile"
}

## Run this function to move transactions from the file in the argument to the "$MAIN_JOURNAL"
movetransactions_function "$FUTURE_JOURNAL"
movetransactions_function "$PREORDER_JOURNAL"
#-------------

#-------------
## sort main.journal (dates and other stuff (please test for yourself...))
## look at https://hledger.org/1.34/hledger.html#print-explicitness to see is you desire this behavior
## README FROM THE HLEDGER MANUAL:
## PRINT PRESERVES EVERYTHING WITHIN TRANSACTIONS BUT LOSES DIRECTIVES AND INTER-TRANSACTION COMMENTS
## THIS IS ONE OF THE REASONS WHY I USE A config.journal !!! and the "$MAIN_JOURNAL" only contains transactions!
hledger print -x -f $MAIN_JOURNAL -o $MAIN_JOURNAL

## This is an alternative way to sort the main journal without relying on hledger -o (which may or may not work correctly...)
# tmpfile=$(mktemp) &&\
#   hledger print -x  -f "$MAIN_JOURNAL" > "$tmpfile" &&\
#   cp "$tmpfile" "$MAIN_JOURNAL" ;\
# rm "$tmpfile"


## Alternative way to sort the "$MAIN_JOURNAL", here it is a dry run
#hledger import --dry $MAIN_JOURNAL | hledger -f- print $MAIN_JOURNAL

#-------------
1 Like

I condensed and highlighted it for easier comprehension.
(Warning: don't run this one ! Important comments deleted.)

MAIN_JOURNAL="$HOME/hledger/main.journal"
LEDGER_RECURRING="$HOME/hledger/forecast.journal"
IMPORT_LEDGER_RECURRING="$HOME/hledger/imports/forecast.journal"
PERIODIC_JOURNAL="$HOME/hledger/periodic.journal"
PREORDER_JOURNAL="$HOME/hledger/preorder.journal"
FUTURE_JOURNAL="$HOME/hledger/future.journal"
IMPORT_DATE=1day

hledger print -f "$PERIODIC_JOURNAL" --forecast --begin=1day --end=10year -o "$LEDGER_RECURRING"

create-forecast_function () {
  hledger print -f "$1" --forecast --begin=-1year --end=1day -o "$2"
}

create-forecast_function "$PERIODIC_JOURNAL" "$IMPORT_LEDGER_RECURRING"

importtransactions_function () {
  hledger import -f "$MAIN_JOURNAL" "$1"
  echo -e "\n\n"
}

importtransactions_function "$IMPORT_LEDGER_RECURRING"

movetransactions_function () {
  tmpfile=$(mktemp) &&\
    hledger print -e="$IMPORT_DATE" -f "$1" > "$tmpfile" &&\
    hledger import -f "$MAIN_JOURNAL" "$tmpfile" &&\
    echo "Import from: $1" &&\
    echo -e "\n\n" &&\
    hledger print -b="$IMPORT_DATE" -f "$1" -o "$1" ;\
  rm "$tmpfile"
}

movetransactions_function "$FUTURE_JOURNAL"
movetransactions_function "$PREORDER_JOURNAL"

hledger print -x -f $MAIN_JOURNAL -o $MAIN_JOURNAL
1 Like