2015-08-15 13:37:17 UTC
I've finally had some time to explore Nick's view-models branch  on
master. I think that this is the way forward; let me explain why.
Currently and for some time, all tree views in Gramps work through a
complicated set of callbacks. The idea is that instead of loading the
entire set of data into memory (the model), it will load and unload only
the visible data. Thus, as you hover your mouse, or scroll up and down, the
database is being constantly queried to show the visible data.
Nick's prototype removes those complications, trading a longer start up
time (all data is loaded into the model first) for no further
activity---there is no reason for any callbacks as you view the data.
The original choice was fine at the time: Gtk models/trees weren't very
good, computers didn't typically have as much memory as they do now, and we
had a single, local database connection. All of these are now different.
1. I think that there are bugs in the current system; it has a
Least-Recently-Used (LRU) cache that is supposed to prevent further
database access, but I see a lot of activity. But even with that, there
would be a lot of code running as you hover/scroll. Nick's prototype
removes all of this.
2. We can optimize these model loads at the DB-level. Currently, we must
access BSDDB one record at a time. But with other backends, we can get
groups of data with one db access. So, we could add methods like
db.load_person_model(model) to the database. It wouldn't change the access
time for BSDDB, but others could substantially be sped up.
3. It may be the case that loading all of the data into memory might not be
a good/viable solution for everyone (limited memory) or for certain sizes
(millions of records). We can have a max-count option over which we could
present the view in "pages" so that we only load a subset of the data, with
"prev" and "next" buttons. We could also have a "select" column to keep
track of selected items no longer on the current page.
4. I turned on some messaging in master with Nick's prototype whenever the
database was accessed. Briefly, we access the database way too often,
sometimes repeatedly for the same information. We have treated db access as
if it were free, and it isn't, even when it is pretty fast. Removing these
unnecessary queries will make the system faster, for all backends.
5. Nick's prototype isn't done---there are still some linear scans that we
can improve on, either with better code, or with additional database
indexes. For example, we don't need to scan the full people table for a
list of surnames, we can get that from db.get_surname_list().
This is a big change, and one that we can keep refining. (The two-pane
views are interesting, but that is not the main point.) I've brought the
branch over to a master branch .
Let's merge this and keep refining.
 - https://github.com/Nick-Hall/gramps/tree/view-models
 - https://github.com/gramps-project/gramps/tree/view-models