This page is intended to stimulate some discussion about developing utilities for RootsMagic in a high level programming language. Please add your thoughts. Would you be interested in:
- using utilities that do things RootsMagic does not currently do?
- developing such utilities?
- testing?
Development Platform?
I had been setting up an old Borland Delphi 4 with DiSQLite3, thinking it was time to move some of the SQlite scripts into some kind of menu driven collection of utilities to make them more amenable to persons flummoxed by working directly with queries. I had just compiled a fake RMNOCASE extension for SQLiteSpy on it which would also work with DiSQLite3, and thought that this would be a logical platform because RootsMagic is also developed on contemporary Delphi and DiSQlite3.
Then along came Steve Turley who started the ball rolling with RmSplit on the page SplitTree, using Visual Studio C# 2010, because, as he said:
I chose Visual C# because I find it's a quicker development environment for projects like this. The advantage of C++ is that it has a more direct connection to the base sqlite library which is written using unmanaged code. For other projects, C++ tends to be more portable, but an application written specifically for Windows like this one is tied to a single platform anyway. Mostly, it's a personal preference. I am a physics professor and do a lot of research work with students. I have found that I can bring them up to speed faster on C# programming than C++ so I tend to gravitate towards that for simple windows programs. That said, if I was writing a big production code or if performance were at a premium, I'd probably use C++.
I installed Visual C# 2010 Express and Steve’s project package and was able to generate my own version of RmSplit with minor changes in the user interface. I’ve been a program ‘dabbler’ since before DOS, just enough to whet my appetite and maybe accomplish something odd for work or play but never on an ongoing basis. Never got into object-oriented programming – I’m a procedural guy and not sure if C# is the way to go for me.
Another option for me is Tcl/Tk. Tcl was used to develop SQlite and I have some passing acquaintance with it from about a decade ago. And it’s totally free, unlike DiSQLite3 and Delphi, and, for that matter, anything above the Express level in Visual Studio products. On opening Visual C# Express, it says it is for Evaluation Purposes Only; I cannot find the EULA so am unsure of the restrictions, and am seeing a 28 day countdown which suggests that there may be features that get turned off when that period expires.
PHP has SQLite built-in, is well suited for server-side web applications, and is free. Never used it and don’t know how suited it is to a stand-alone app.
And I just had a brief look at SQLite for Excel. Took me a while to understand what the sample demo was doing. Basically adds VBA macros to a workbook via which a SQLite database can be worked on. The demo seemed pretty slow as does RmSplit so I wonder about the number of layers between the user and the data.
The worrisome thing is the learning curve on any of these. Peter Norvig, author of Teach Yourself Programming in Ten Years says his “recommendations for a first programming language would be Python or Scheme. SQlite is included in recent Python distributions; links for Scheme wrappers on sqlite.org are dead.
Peter also advises to Use your friends, i.e., use what they use as there is greater strength in a community. If you are interested in collaborating on the development of utilities for RootsMagic, what would you use? What are you using?
(Added by Jerry Bryan): I’ve been following this discussion with great interest. I agree with the opinion that as long as SQLite remains unbundled where a user has to download an SQLite manager and also download an SQLite script and figure out how to operate the two together, SQLite scripts will only reach a very small niche of the RM4/5 user base. Being able to download a bundled program which is fully self contained (an EXE file, if you will) should enable useful utility functions to reach a much greater audience than the unbundled approach will ever reach. So I’m excited about the possibility of bundled utility applications.
I’m an old C/C++ programmer from way back and I know nothing about C# except in as much as C# may have common roots in C. I’m not interested in advocating for C/C++ as the platform of choice, but I am interested possibly in writing a RM4/5 utility application myself in C/C++. And I’m interested in seeing a “baby steps” example of what’s required. Exactly what C/C++ library for SQLite do I have to download and from where? How do I do the compile? And would it be possible to have a really simple examplar to look at, even if it were written in C# instead of C++? For example, I would love to see a C++ or a C# program that would do something so simple as SELECT * FROM NameTable, and gather up and print the results. In other words, the example program would need no real logic at all. I would just like to see how to put the pieces together. If I could get even one such simple program to compile and execute, I think I could easily go from there to developing much more complicated and useful applications.
(Added by Steve Turley)
Here is an example in C++ to supplement the C# version posted elsewhere. My two easy choices are using gcc under linux or Visual Studio. I’ll start with Visual Studio since I’m guessing an RM user is probably running on a Windows platform. By the way, there is one issue that doesn’t show up in a query like SELECT * FROM NameTable. If you want to modify a table in the database, you may have to worry about the collation. Many of the tables are index with an RMNOCASE collation. I’ll save that issue for a little later.
#include <iostream>
#include "sqlite3.h"
using namespace std;
static int callback(void *unused, int argc, char **argv, char **colName){
for(int i=0; i<argc; i++){
cout<<colName[i]<<" = "<< (argv[i] ? argv[i] : "NULL") <<endl;
}
cout<<endl;
return 0;
}
int main(){
cout<<"Example RM Database Queryn";
sqlite3 *db;
int rc = sqlite3_open("test.rmgc", &db);
if(rc){
cerr<<"Error opening database filen";
return 1;
}
cout<<"Database table opened successfullyn";
const char *sql = "SELECT * FROM NameTable LIMIT 20";
char *error=NULL;
rc = sqlite3_exec(db, sql, callback, NULL, &error);
if(rc){
cerr<<"Error executing queryn";
sqlite3_free(error);
}
sqlite3_close(db);
system("PAUSE");
}
To set this up with the sqlite3 library you need to following these steps:
- Create the above program as an unmanaged Win32 console application.
- Download the source code for the sqlite library at http://sqlite.org/sqlite-dll-win32-x86-3070900.zip
- Unzip this file to the course code directory for the above program. (An easy way to find this directory is to right click on the tab for this code and choose open containing folder.)
- Add the sqlite3.def and sqlite3.dll to your project directory. (Right click on the project name and choose Add/Existing item…)
- Create sqlite3.lib by executing the VS command LIB /DEF:sqlite3.def.
- Add sqlite3.lib and sqlite3.def to the list of library files and Module definition files used by the linker.
- Right click on the project name and choose properties.
- Select the Input item in the Linker Group.
- Add sqlite3.lib to the list of files with a terminating semicolon.
- Add sqlite3.def as the Module Definition file.
- Download the file http://sqlite.org/sqlite-dll-win32-x86-3070900.zip
- Unpack the file sqlite3.h and put it in the header folder for the project.
Here is a zip file with my project in it as I built it. RmCppConsole.zip
(Added by Jerry Bryan)
First a procedural question/comment: Simply editing one of the wiki pages seems not to be the best way to carry on what is essentially a threaded conversation. Is there a better way to carry on such conversations within the confines of the wiki architecture? Like most people, I’ve read wikis for years, but this RM/SQLite wiki is the first time I’ve ever joined a wiki and posted to it. I’m more familiar with threaded conversations in the context of forums and bulletin boards, blogs, and even old fashioned E-mail.
Second: Much thanks to Steve Turley for posting the sample code. It’s exactly what I was looking for, even down to the fact that it’s a console application instead of a Windows application. I’m a computer science guy, but I do IT for a living and seldom have an opportunity to do real computer science. I do teach a course occasionally, but it’s been several years since I’ve even had time to do that. Being a college administrator is not as much fun as teaching or as doing real computer science. So developing some object oriented code on sort of a hobbyist basis sounds like a lot of fun.
Third: As to my experience with C++, it’s strictly as a hobbyist. Our college still uses C for its intro and data structures classes, which are the ones I usually teach if I teach. The question of “first language to teach” has long been a hot topic of debate among computer science, and our college has chosen to stick with a procedural language as the “first language to teach”. I find that my C students always ask what the difference is between C and C++. It’s a curiously difficult question to answer in a way that’s both brief and meaningful. You really need to experience object oriented programming by doing it, and most people learning an object oriented language have a major “aha moment” when they “get it” about what object oriented programming really is. Having said that, I’ve probably written about 40,000 or 50,000 lines of C++ code in the last 10 or 15 years (just for fun). Until very recently, I’ve used the (now) very old Borland C++ compiler version 5. Serendipitously, I’ve just started the switchover to Visual Studio Express. I haven’t decided yet if the Express version is good enough, or if I’m going to need to upgrade to a professional version. For my ongoing projects that are not RM/SQLite related, I still don’t need anything except for console mode, but I’m eventually going to need huge data spaces and 64 bit operation. It’s not clear to me if Visual Studio Express will do 64 bit operation, or if it’s limited to 32 bit. Of course, 32 bit is fine for any RM/SQLite needs. And by the way and in case anybody is interested, my big C++ project for years has been the investigation of the mathematics of Rubik’s cube. There are many interesting problems that are computationally intensive and that sometimes run for months on a PC.
Discussions & comments from Wikispaces site
VB.net
08 January 2012 05:25:52
What would be the advantages and disadvantages of VisualBasic.net?
rsturley
Bundled Utilities
08 January 2012 05:34:22
The idea of bundled utilities intrigues me. My RmSplit was a bit of an experiment thinking about a larger package of utilities. Here are my comments on some of your ideas on the best development language.
C#: a proprietary environment that only works on Windows. It sounds like Bruce is working on expanding RM to be available on several mobile platforms and Mac OS. In principle, we could extend a project to work on other platforms using mono, but that would probably not be ideal.
Python: I’ve tried some of this kind of work on python before. It is a great tool for one-up kinds of things, but it doesn’t have strong tools for a GUI that would be a good interface for non-experts.
PHP: I have used this as a web programming language and a stand-along scripting language. I think I like this suggestion the best of the ones you suggested in your page. One nice thing about it is that the project could be hosted “in the cloud” and we don’t have to worry about distributing or updating the code. We could have a web-based package that incorporated an organized set up the SQL scripts appearing in this wiki process them on the server rather than the user’s local computer. I don’t think the community of users is so big that it would be a processing problem, but we could try it on a small set of scripts and see how much traffic we get.
Java (one you didn’t mention): This would either be stand-alone code (more portable than C# probably) or server based (.jsp files which similar organization to PHP). If we go with a cloud-based solution, I prefer PHP over java because server maintenance is a bit easier and servers supporting PHP are more widespread than those supporting jsp.
Anyway, if you decide to support a language and platform I have experience with I’m willing to help.
LessTX
09 January 2012 18:26:47
Just for testing and looksee:
https://leslietest.slsapp.com/
https://leslietest.basecamphq.com/
The first is a springloops site, and it has a link of some kind to basecamp, so I created one of those, too.
That could be a place for us to start out…
the springloops site for the codebase stuff, and the basecamp for the visual planning, conversations, ideas, chat, milestones, etc
Are those sites and utilities that you think folks would find easy enough to use?
ve3meo
RMNOCASE
25 January 2012 18:45:03
Blundering around in Steve’s RMsplit project, trying to learn the rudiments of C#, OOP, Visual Studio and wanting to leap ahead to something useful, I did manage to run the SQLite Integrity Check with the fake RMNOCASE Steve came up with. The outcome is not good. It reports many index errors on databases that RM5’s integrity check reports as OK, just as does SQLiteSpy with the fake extension built with Delphi. The latter has many more reported discrepancies and the two do agree on most of those reported by C#.
This means that, until we can use the identical collation sequence as in RM, procedures that modify data so collated should be precluded from being used on RM4 databases and that, for RM5 databases, a strong warning that there may be unanticipated consequences and advise the user to use the RM5 Rebuild Indexes feature after any such changes by the utility.
ve3meo
Inline comment: “I would love to see a C++ or a C# program that would do something so simple as SELECT * FROM NameTable”
04 September 2018 03:39:33
ve3meo Jan 9, 2012
Jerry, download the project file that Steve posted on SplitTtree and look at RmSplitForm.cs for examples of the query. He also includes in the project the necessary SQLite files.
rsturley Jan 9, 2012
Jerry,
I created a c+ example program that does the simple thing you asked about (SELECT * FROM NameTable) using Visual C++. To keep it simple I didn’t put in any GUI stuff, but rather chose a simple console application. The example and a brief explanation of the process of creating it are in the wiki page.
ve3meo
Inline comment: “Is there a better way to carry on such conversations within the confines of the wiki architecture?”
04 September 2018 03:41:10
ve3meo Jan 12, 2012
Yes! Each wiki page has a Discussion “forum”, i.e., a page under the wiki page where you can post a topic (really any topic but preferably one related to the subject of the wiki page) and a threaded (sort of) discussion can flow. By sort of threaded, I mean that all posts under a given topic can be found under that topic but you cannot see which post a post has replied to and the reply system includes no quotes. For a straight serial conversation it’s fine. To open the Discussion page, you click on the double bubble button on the wiki page.
A variant of the Discussion page is to comment on selected text on the wiki page itself, as I am doing here. You have to Edit the page, highlight the text to comment on, click the Comment bubble button and write your comment in the form. The comment also appears on the Discussion page under the topic which is the highlighted text and replies thereto automatically show up as comments on the wiki page itself. To see the comments on the wiki page, you click on the button with the single bubble on the wiki page.
ve3meo Jan 12, 2012
OOps, got the way the topic works wrong, as you can see… The highlighted text on the wiki page is quoted in the post seen on the Discussion page; the topic of the post is “Comment added:”+ first few words of first sentence of comment.
rsturley
08 January 2012 05:38:42
VB.net would have the same interface to SQLite and WindowsForms environment as C#. Personally, I think the C# environment is richer and better suited to collaborative projects (easier to define and enforce interfaces using classes). However, the community of programmers with an interest in this project may have a larger group with VB.net background. The end users of the utilities probably won’t see much difference in applications programmed either way.