Direct Downloads
Do you have a working importer that smoothly extracts data from your statements in from a downloaded OFX/CSV/other file? The next step is automating the download itself. Quoting from the ofxtools webpage:
OFX data is widely provided by financial institutions so that their customers can import transactions into financial management software such as Quicken, Microsoft Money, or GnuCash.
Retrieving Via ofxget
We use ofxget, (part of ofxtools) to
automatically login and retrieve our OFX file for supporting institutions, and place it
in the staging folder. When not supported, we fall back on using our browser to manually
download files into the staging folder (~/Downloads) for further processing.
Not all institutions support this download API. This will depend on where you live
and whom you bank with. A little over half (35) of my
61 accounts support retrieval via ofxget. Most (but not all) my
[[
Being Smart About Which Accounts to Update When|most frequently downloaded accounts
]] support it.
For the ones that support it, a few parameters are necessary to access the web API. These are typically known by commercial vendors such as Quicken. How do folks like us get access to it? OFX Home is a peer-driven website that “maintains an extensive directory of financial institution data.”
The cool thing about ofxget is, it can look up these parameters from OFXhome for your
bank. Read the well written “Using ofxget- TL;DR” page
here.
Scripting Your Downloads
I use bean-download, which is a command installed as a part of
beancount-reds-importers, as
a part of my workflow to accomplish one of two things for each account:
- sites supporting OFX downloads: login and download an OFX into my staging area (~/Downloads)
- for institutions that don’t support direct download, you can simply place a text reminder to yourself of any sort. For example, I place a link to remind myself of how to obtain Amazon purchases, or reminders about site-specific quirks.
Though it’s a simple tool, I use it very frequently, and hence, it has all the bells and whistles:
- multi-threaded downloads
- classify your sites into arbitrary buckets and download just the bucket (eg: just credit cards)
- full command line auto completion support, including your configured sites and buckets
- progress bar, color output, etc.
Configuring bean-download
I list all my accounts in a download.cfg file thus:
[DEFAULT]
ofx_pre = pass show dummy > /dev/null; ofxget stmt --nokeyring -u
[tdameritrade_johndoe]
type = ofxget
cmd = %(ofx_pre)s username --password $(pass beancount/tdameritrade_johndoe) ameritrade-alt -i 012-345678 > ~/Downloads/tdameritrade_johndoe.ofx
# manual
[discover_johndoe]
type = manual
display = "Download 'all year' ofx (else, no ofx option appears)"
Running bean-download config-template will also generate the example above, which you
can modify according to your needs.
bean-download list-institutions
Downloading using bean-download
Simply typing:
bean-download download
will download, in parallel, all the latest ofx files from all supporting institutions. This only takes a few seconds.
bean-download download --institutions citi,fidelity
will download the latest ofx files for the specified institutions.
bean-download can autocomplete what comes after --institutions above in bash, zsh,
and fish shells, by picking up the institutions in your config file. See
here for
how to configure this.
Here is how the whole thing looks:
Of course, the downloads in the demo failed since real credentials were not used.
See this article for how to process the downloaded files.
-
You will need this patch (see this thread) until it is released ↩