tblancher's timetracker timekeeping system
Go to file
tblancher 1b281742dd Spelling and list formatting fixes 2020-04-07 20:39:42 -04:00
images Added INSTALL.md and related images, with mention in README.md 2020-04-07 18:44:01 -04:00
.gitignore Cleaned up organization handling, added to .gitignore 2018-10-19 10:04:23 -04:00
.vimrc Made vim more standard 2020-04-07 17:33:56 -04:00
INSTALL.md Spelling and list formatting fixes 2020-04-07 20:39:42 -04:00
README.md Fixed INSTALL.md link 2020-04-07 18:48:18 -04:00
chug.sh Updated README for `chug.sh -i` and fixed log handling 2020-04-02 17:03:42 -04:00
do_process.sh Removed unnecessary call to xargs 2020-04-03 17:06:41 -04:00
month-pack.sh Rename {month,year}-pack with .sh filetype suffix 2020-04-02 16:18:09 -04:00
test.py Cleaned up organization handling, added to .gitignore 2018-10-19 10:04:23 -04:00
test.sh Cleaned up organization handling, added to .gitignore 2018-10-19 10:04:23 -04:00
timetracker-sections.pl Cleaned up organization handling, added to .gitignore 2018-10-19 10:04:23 -04:00
timetracker.kmmacros Added more KM macros 2020-03-12 11:42:57 -04:00
timetracker.py Modified for short tasks (less than 288s/0.08 hours) 2019-08-09 16:57:28 -04:00
timetracker.vim Added alternative macro for vim 2019-06-14 11:18:14 -04:00
year-pack.sh Rename {month,year}-pack with .sh filetype suffix 2020-04-02 16:18:09 -04:00

README.md

timetracker

tblancher's CLI-based time tracking system (see demo videos of the timetracker setup and Clarizen process)

INSTALLATION

See INSTALL.md in this repository.

OVERVIEW

This framework is what I use to keep track of my time, as I work. Only one task is allowed concurrently, so if I switch to another task, I must end the current task before proceeding. This is tracked in a log file with the following format:

log file format

2017-12-08 08:21:27:  Begin [Productive Non-billable] email triage
2017-12-08 08:59:05:  End   [Productive Non-billable] email triage
2017-12-08 08:59:06:  Begin [CS Support] internal task 1
2017-12-08 09:10:55:  End   [CS Support] internal task 1
2017-12-08 09:10:56:  Begin [Productive Non-billable] internal task 1
2017-12-08 09:15:14:  End   [Productive Non-billable] internal task 1
2017-12-08 09:15:14:  Begin [Customer 1] task 1
2017-12-08 09:19:52:  End   [Customer 1] task 1
2017-12-08 09:19:52:  Begin [Customer 1] PDROP-0000000 case-related task 1
2017-12-08 09:27:37:  End   [Customer 1] PDROP-0000000 case-related task 1
2017-12-08 09:27:38:  Begin [Customer 2] task 2
2017-12-08 11:00:39:  End   [Customer 2] task 2
2017-12-08 11:00:40:  Begin [Customer 1] PDROP-0000000 case-related task 2
2017-12-08 11:07:24:  End   [Customer 1] PDROP-0000000 case-related task 2
2017-12-08 11:08:56:  Begin [CS Support] internal task 2
2017-12-08 11:25:43:  End   [CS Support] internal task 2
2017-12-08 12:47:50:  Begin [Customer 1] PDROP-0000000 case-related task 1
2017-12-08 13:31:52:  End   [Customer 1] PDROP-0000000 case-related task 1
2017-12-08 13:31:52:  Begin [Customer 3] external meeting
2017-12-08 14:11:13:  End   [Customer 3] external meeting
2017-12-09 14:11:14:  Begin [CS Support] documentation creation
2017-12-09 17:01:47:  End   [CS Support] documentation creation

The timestamp is in most-to-least significant units, though any valid date and time should work (however, any other format is untested. BEWARE!). The separator between the timestamp and the Begin or End keywoards is : , literally a colon followed by two spaces. Anything else will break timetracker.py, and the rest of it will fall down. Each task must have a Begin and End line, or else the output of timetracker will not be correct. Too many begins or ends should be detected by timetracker.py. The name of the task is free form, and must be identical for the Begin and End (otherwise timetracker.py will think they're different tasks).

Certain strings have special meaning to timetracker.py. The category name in square brackets specifies that the task is related to a specific customer, or should be tracked on a certain item in Clarizen, to make transferring to Clarizen more straightforward. These can be any string, even with spaces, but beware of using shell special characters. The names can be anything, so things like [BB&T], [Citi], [Wells Fargo], or [CS Support] are perfectly valid. For the author's most common categories, macros are set up with text string triggers in Keyboard Maestro (more on that below).

timetracker.py usage

The timetracker.py script is the basis, it tallies the time for each task and then outputs a report on how much each task takes to complete. It takes as its argument one or more text files, the log files in the format above. It should only report one line item per task, regardless of how many times it appears in the log. Each task should have a category/tag in square brackets, the behavior of do_process.sh (see below) is now undefined if the tag is left out.

Note that each task now gets rounded up to the nearest quarter hour (15minutes), unless the task takes less than 0.08 hours (288 seconds). Every documented task will take at least 0.08hours/288seconds. If the task takes 289seconds, it is automatically rounded up to the nearest 15minute increment. If a task takes 30 minutes plus one second to perform, it's rounded up to 0.75hours. Management is OK with this rounding scheme, we may revisit it later if it becomes problematic.

The output looks like this:

timetracker output

[CS Support] documentation creation                    3.00hrs
[CS Support] internal task 1                           0.25hrs
[CS Support] internal task 2                           0.50hrs
[Customer 1] PDROP-0000000 case-related task 1         1.00hrs
[Customer 1] PDROP-0000000 case-related task 2         0.25hrs
[Customer 1] task 1                                    0.08hrs
[Customer 2] task 2                                    1.75hrs
[Customer 3] external meeting                          0.75hrs
[Productive Non-billable] email triage                 0.75hrs
[Productive Non-billable] internal task 1              0.08hrs

Section total:          8.41hrs

You may have noticed, the output of timetracker.py is in alphabetical order by category, then by task. This follows for ./do_process.sh and ./chug.sh below.

do_process.sh

This script filters the output of timetracker.py, giving each category (in square brackets) its own section, with its own tallies. The header is the list of categories and the log file basename (below this is example, but would normally be a date such as 2017-12-08). After the header, each line of output and sections are designed to be directly transferred to Clarizen manually, and the tally used to verify the daily and weekly totals in Clarizen. It calculates the time total for each organization/category. At the end it prints a grand total for the day, which should be used to cross-verify in Clarizen.

[CS Support]
[Customer 1]
[Customer 2]
[Customer 3]
[Productive Non-billable]
--
example



[CS Support] documentation creation                    3.00hrs
[CS Support] internal task 1                           0.25hrs
[CS Support] internal task 2                           0.50hrs

Section total:          3.75hrs

[Customer 1] PDROP-0000000 case-related task 1         1.00hrs
[Customer 1] PDROP-0000000 case-related task 2         0.25hrs
[Customer 1] task 1                                    0.08hrs

Section total:          1.33hrs


[Customer 2] task 2                                    1.75hrs

Section total:          1.75hrs


[Customer 3] external meeting                          0.75hrs

Section total:          0.75hrs


[Productive Non-billable] email triage                 0.75hrs
[Productive Non-billable] internal task 1              0.08hrs

Section total:          0.83hrs



Grand total:          8.41hrs

It takes as its argument a filename with the current date log (date +%F format, such as ./do_process.sh 2020-03-12.log), or it assumes the current date log file. Also, arbitrary filenames can be passed, so example.log becomes ./do_process.sh example.log.

Projects in Clarizen are listed alphabetically, so to transfer data from the ./do_process.sh output you simply copy the data lines from the ouput put, and paste it into the notes section of the Clarizen entry. Enter the duration of the task at the top of the Clarizen entry, and select the category and subcategory of the entry.

chug.sh

This script is designed to be run on Mondays, after the previous week of log files have been generated and closed out. The standard Monday usage takes no arguments, it expects all log files to be processed to be in the current directory. It runs do_process.sh once for each day of the previous week, cleanly skipping any log files which do not exist. It pauses after each day report is output, allowing the user to transfer the times manually to Clarizen.

./chug.sh takes a single optional argument, a week offset (in case ./chug.sh is executed for log files further back than last week). This uses the GNU date functionality of calculating "last Monday." On Monday this will be "today - 7 days", but on the following Tuesday this will evaluate to "yesterday". If Monday is a holiday and you're entering your timesheets on Tuesday you can enter ./chug.sh 1 and it should do the right thing. If it's the first Monday of the month and you need to process the previous four weeks of logs, use ./chug.sh 4. The output of ./do_process.sh prints the date that is being processed at the top, if your incantation of ./chug.sh is wrong, you can quit and adjust accordingly. ./chug.sh with no arguments is equivalent to ./chug.sh 0.

The output is the output of ./do_process.sh piped to less for each day, pausing so the user can go through that day's output and transfer the items to Clarizen. If no time was logged for a given day (the file does not exist), ./chug.sh prints the missing date, but otherwise silently skips it. All seven days of the week are processed, Monday through Sunday.

If you're only interested in the weekly summary (and don't want the output of ./do_process.sh <date>.log | less), you can pass the -i option to ./chug.sh, like so: ./chug -i, or if you need a prior week, e.g. ./chug -i 2. The output will look similar to the following:

2020-03-16
9.05

2020-03-17
10.90

2020-03-18
11.96

2020-03-19
7.50

2020-03-20
9.38

2020-03-21
0

2020-03-22
0

Weekly Total:  48.79

month-pack.sh

timetracker.py and the related do_process.sh and chug.sh scripts are designed to have each day with its own YYYY-MM-DD.log file in the current, timetracker directory. Over time, the log files in this directory can become quite numerous and unwieldy. To help combat this, month-pack.sh takes all the log files from the previous month, adds them to a compressed tarball, and deletes them from the directory. It is designed to be run once all of the log files for the previous month have been processed into Clarizen.

year-pack.sh

In the same vein as month-pack.sh, year-pack.sh tars up all the monthly tarballs (named YYYY-MM.tar.xz), and puts them into a single YYYY.tar file. It is designed to be run in January when all of the previous December log files have been processed. All of the YYYY-MM.tar.xz files will be deleted once the YYYY.tar file is created.

CAVEATS

Tallies

Since I (the author, Trey Blancher) have been using this system to keep track of time, I've noticed that sometimes either Clarizen or these scripts get slightly off. Usually it's no more than 0.25 hours off in the tallies, but it gets time consuming trying to track down where the tally went wrong. If I do find the culprit, it's usually because I've entered the wrong time for a specific task in Clarizen (i.e., entering '0.08' instead of '0.25' for some tasks).

vim

The vim-specific files in this repository are tailored for my tastes. One big item of note, I've disabled vim cursor navigation with the arrow keys (Up, Down, Left, Right), to force me to get into the habit of using h, j, k, l for cursor navigation, and only navigate in normal mode (not insert mode). You will probably want to delete the following lines from .vimrc if you're not interested in the true Vim way®:

" Force me to stop using arrow keys
inoremap <Left> <Nop>
nnoremap <Left> <Nop>
inoremap <Right> <Nop>
nnoremap <Right> <Nop>
inoremap <Up> <Nop>
nnoremap <Up> <Nop>
inoremap <Down> <Nop>
nnoremap <Down> <Nop>

My Keyboard Maestro macros for timetracker make use of specific vim macros, and certain keybindings within .vimrc. The file timetracker.vim contains a couple of alternate macros for this purpose.

Several of the vim shortcuts I've programmed assist in managing the log text file. Invariably there will come a time where I forget to document a few tasks right away, so I piece together the logs from Slack and email, and edit the logs so it matches the format. One notable "normal" mode mapping is 'Y', which copies from the cursor position to the end of the line ('yanks' from the current position to the EOL, similar to 'C' or 'D' for changing or deleting to the EOL). It is equivalent to the action 'y$', but is only one keystroke instead of two. The Keyboard Maestro macro for 'End' tasks uses this mapping heavily.

Keyboard Maestro

Much of creating the log entries depends on specific Keyboard Maestro macros the author has set up. The file timetracker.kmmacros contains all of the macros used for this. The same text string trigger (Command + Shift + D, but you can assign it to any trigger) brings up a menu (known as a "conflict dialog" in KM), where I select 'Begin' or 'End'. The 'Begin' macro prints a log line using the current timestamp, like so:

2020-03-11 19:14:42:  Begin [CS Support] timetracker doc

The 'End' macro copies the category and task name from the previous 'Begin' macro, to cut down on the amount of typing (and eliminate the possibility of typos). It then prints another 'Begin' line, ready for the next task. The output using the line above would look something like this:

2020-03-11 19:14:42:  Begin [CS Support] timetracker doc
2020-03-11 19:30:36:  End   [CS Support] timetracker doc
2020-03-11 19:30:36:  Begin 

I also have Keyboard Maestro text string triggers for common categories, to minimize typing. For instance, the text string triggers [css becomes [CS Support] , [non becomes [Productive Non-billable] , [mm or [MM becomes [Mass Mutual] , [vzw becomes [Verizon Wireless] , [pin becomes [Company] (to match the Pindrop related items in Clarizen), etc. These are included in timetracker.kmmacros for your convenience.