mongodb and base

users
#1

I don't want to try and tell you your business, but I had a few
thoughts on your mail:

No problems. You have some very good feedback which I appreciate and I couldn't say much because I only had the roughest of conversations with this client. It's still not fully fleshed out but, there is more political issues than technical.

>
> > Do you have use cases?
> I'm preping to talk to a real estate title authority.  they have
> 60+years of title info in the area and it is all held in informix.

That's a lot of data. And Informix is a real workhorse of a database.
Arbitrarily shifting to MongoDB and Base just doesn't sound like a well
though out decision...

It was more exploratory thought. Informix is an extremely expensive database, IBM stripped out essential functionality and the current database is running on RedHat 5 and poorly maintained. Informix is similarly old and they will not upgrade. Its been reported that the cost of the Informix database plus any number of third party report writers would cost them 10 years of revenue.

I have a lot more conversations to go before I can validate their financial assessments of the conversion.

Base doesn't strike me as a rock solid, "serious" solution, but that
might be just my perception.

Probably is it. I was trying to explore options for two reasons. I should have been clear about separating out the two.

However, there are a few more pointers in

your mail that suggest the decision may be less than considered:

> they generate reports on the fly (hence base) and would like to be

What exactly do you mean by this? That they regularly want custom
designed reports? Sounds odd. Surely they must have some idea of what
they want to get out of the system on a regular basis? Otherwise it
implies that they don't really know what they want the system for. And
that right there would be a fundamental problem, one that no amount of
engineering will solve.

That solution is unfortunately locked up in political challenges that will not be resolved until 1 of the lawyers retires. But from what I've seen, there is a significant need for flexibility in report generation. How much is a real can't be revealed until they let me understand more about how they do their work.

> able to add information on properties incrementally without always
> rewriting the schema and go through the database conversion.

A common request, but almost always a bad idea. By "adding information"
you must mean adding fields to the database, which should always be a
carefully considered decision, not an "as the mood strikes" decision.

Given the nature the information and their standing as a title authority, I can assure you that the rate of change for record editions is very slow and considered.

Assuming that such fields are purely additional data, and in no way
form part of any relationships, then sure, they can be added quite
easily, but the question is always who is going to go through all the
current entries in the database filling in that new field? One can make
the new fields nullable, and just let all old data have nulls for those
new fields, but that just leaves a mess of incomplete data in the
database. If they've been collecting data for the past 60+ years, they
must have a pretty good idea of what data they need. Real estate data
can't change that often.

From what I can gather, you're mostly right. The primary change is our building together more detailed information about properties that we had in the past and do it more accurately. I think they're also changing legal requirements on titles with regards to title searches and their validity. Fortunately, those changes are very very slow.

> Yes, there may some legal reasons why they can't do this but htat is
> why er are talking.

Legal reasons? I can't think of any. This is their data, they can do
what they like with it, I should think. But I'm a technical guy, not a
legal guy, so I really have little idea. All I *can* tell you is that
from a technical standpoint this sounds like a nightmare waiting to
bite you in the posterior later. And the fact that you think there may
be legal issues may indicate a misunderstanding about what it is the
client is trying to do.

When it comes to this kind of information, there are all sorts of weird legal quirks built up as a result of hundreds of years of additive legal precedent. I know enough about their world there to consider walking away from this contract because of the internal bickering.  , I probably just described the best reason for walking away. :slight_smile:

> personally, I am an order of magnitude more effective with mongo than
> any sql database.  mongo maps better to my mind than sql.  I'd like a
> base connection so I can be more effective in my ad-hoc db tools

At a fundamental level, it sounds like the problem is with the concept,
not the implementation. I'd go back to the drawing board on this one.
Rethink what your client wants to do, and from there use proper tools
to implement it. Much as I am a fan of LO, I just can't recommend Base
as a tool for serious work in the order of 60+ years of data in an
Informix database.

I'm in the process of figuring out what they really want instead of what they tell me what to do. But when it comes down to it, the most common complaint I hear is that it just cost too damn much and they feel a bit ripped off by IBM.

Between license fees and the lost report writing capability that can not be replaced with third party, they are feeling kind of burned and between a rock and a hard place.

But then again, you know what the client wants, I don't. And as you're
the one who has to do the work, you will need to decide how closely you
want to stick with what you're familiar with. And one assumes you're
the guy who has to live with the decisions. So if you want MongoDB and
Base, I'm sure someone will be able to point you in the right
direction. Unfortunately I can't, as I have no experience with MongoDB.

Remember, using MongoDB and base are two separate decisions. I was trying to close out the matrix and see if I could use beast to replace the report and data entry tools now lost to Informix.

I just thought I'd point out what seems like a mess. You might well
have thought it through thoroughly, and this advice may be unwarranted
and unwanted, and if so I apologise, but I felt it better to offer
caution than to let you step blindly into harm's way.

It's entirely your call of course, but, given that I've done these
sorts of systems for a living for quite a few years, your description
raised warnings when parsed through my experience filter, and I felt I
should tell you about it.

No problem, I appreciate the input. Unfortunately, my experience with SQL over the past 30 years has taught me to stay away from SQL as far as absolutely possible. I know most of my problems with SQL come from an impedance mismatch between how I solve problems and how it wants to express solutions. Usually, the group of people I work with leave the database to the very last and isolated. We do that so the rest of the application is not contaminated by SQL isms and we have a nailed down definition of the schema. We don't have hard numbers yet but with MongoDB, we are not getting any of the chaos and heartache that SQL delivers on a regular basis.

I really do wish I SQL would go the way of COBOL.

#2

(Kind of wandering off the mailing list's topic, but...)

[snip]

No problem, I appreciate the input. Unfortunately, my experience
with SQL over the past 30 years has taught me to stay away from SQL
as far as absolutely possible.

It's kind of sounding like your issue isn't "SQL," per se, but the
entire relational model. It is true that, for some applications, the
relational model does not work well.

[snip]

Usually, the group of people I
work with leave the database to the very last and isolated. We do
that so the rest of the application is not contaminated by SQL isms
and we have a nailed down definition of the schema.

*That*, IMO, when you know your backend store is going to be on an
RDBMS, is just asking for grief.

It's been years since I studied theory on software system design
methodologies, but, last I did, it was coming into vogue to first
design *all* of the data elements, and their transformations, then
design the software around that. My last couple major projects I
used such systems and they worked quite well.

We don't have
hard numbers yet but with MongoDB, we are not getting any of the
chaos and heartache that SQL delivers on a regular basis.

I just took a quick look at MongoDB. Looks interesting. But, just
as being locked into knowing only the RDBMS solution results in the
"if all you have is a hammer" syndrome: I'd argue that one could not
*possibly* deliver the best solutions for all scenarios approaching
them from the perspective that "RDBMS' are to be avoided to the
extent possible, and then left to the last minute."

It's like the "big data" question. There are really, really large
datasets that aren't well-suited to common RDBMS', either. At the
opposite end: Tiny databases in embedded systems. Then there are
directories (read-heavy, light on write activity).

I really do wish I SQL would go the way of COBOL.

I would never say "impossible," but it seems unlikely in the extreme.
SQL is simply a human-oriented way of interfacing with an RDBMS, and
RDBMS' are well-suited to any number of solutions requiring
datastores.

I'm glad you raised this issue, however. I had not been aware of
MongoDB. It deserves a close look. Thanks!

Regards,
Jim

#3

Given the nature the information and their standing as a title authority,

I'm making a guess here, but that sentence implies that the firm not
only has to have current boundaries, but also former boundaries of the
property in question, and be able to explain why those boundaries appear
to have, and in some cases really did change.

The primary change is our building together more detailed information

about properties that we had in the past and do it more accurately.

I don't know what country you are in.

Several decades ago, I had the experience of being tasked with finding
the current, legal boundaries of some property the company I worked for
wanted to foreclose on. The boundary line, according to the title deed
was "Westward from Mama's grave to the three elm trees. John's property
begins at the first tree. Jane's property begins at the third tree."
Needless to say, no survey map, or government database said where Mama's
grave was. Equally helpful was the lack of elm trees on any of the
properties in question.

I can imagine the issues a Title company would have, to prove to a
hostile jury that the property being foreclosed upon, and the property
described in the title deed are the same property.

Instead of foreclosing on the property, we sold that mortgage to a sucker.

I think they're also changing legal requirements on titles with

regards to title searches and their validity. Fortunately, those changes
are very very slow.

Doesn't matter how slow they are. When implemented, the relevant data
has to be added to every parcel, and sub-parcel.

(Here's looking at a parcel of land that the Attorney General of the
State of Arizona, and the Attorney General of the State of Utah told a
judge that he has had ten years to determine who originally owned what,
and where it was located, and that that should be enough time to
determine who owned what, and where it was located, and when they owned
it, and he needs to finish that job yesterday.(In all fairness to the
judge, before that property fell into the courts, the organization that
claimed to own it, probably did own it, even if there are no official
records to show how they acquired it, or when they acquired it, or what
they acquired.))

I know enough about their world there to consider walking away from

this contract because of the internal bickering.

I probably just described the best reason for walking away. :slight_smile:

Or the best reason to add a couple of more zeros just to the left of the
decimal point.

I was trying to close out the matrix and see if I could use beast to

replace the report and data entry tools now lost to Informix.

Using Base and MongoDB as an Informix clone will require writing, and
supporting several extensions for Base.

I really do wish I SQL would go the way of COBOL.

Intrinsically, there is nothing wrong with either COBOL or SQL.
The problem is that people who use those tools tend to suffer from the
computing equivalent of extreme monolingualism, thereby demonstrating
the validity of the Sapir-Whorf hypothesis.

jonathon

  * English - detected
  * English

  * English

<javascript:void(0);>

#4

Hi :slight_smile:
I hope that is "Redhat Enterprise Linux" rather than Redhat "hurricane" or
"apollo"!

RHEL-5.11 is just a couple months old and is stable branch. It makes sense
that they wouldn't update it.

Redhat 5.0, code named "hurricane", is from 1997! Even if it's been
patched and updated then it sounds a little worrying. If they are running
their desktops on Win98 too, in order to keep everything from the same era,
then this company sounds increasingly like one that is setting themselves
up for a massive failure. It is possible to buy a really cheap 2nd hand
hard-drive that is far newer than their current set-up, or even find ones
on skips or landfill sites. Then their techies could install a newer
version of CentOS (ie drop-in replacement for RHEL except without the
paid-for support) as a dual-boot ready to switch over when their ancient
system falls over. If it is the non-RHEL system then they might be paying
a monthly or annual fee for support that Redhat can't give any more!

I'm guessing they aren't complete morons though so it's gotta be RHEL-5.11,
which makes them VERY current!
Regards from
Tom :slight_smile:

#5

RHEL-5.11 is just a couple months old and is stable branch. It makes sense
that they wouldn't update it.

Redhat 5.0, code named "hurricane", is from 1997! Even if it's been
patched and updated then it sounds a little worrying.

...

I don't think it's quite from 1997 but definitely early to mid 2000's. if I remember correctly it was conversion from SCO. Two forces at work here. specifically lost functionality and expense to regain most of that lost functionality.

  then their techies

It's a bad assumption to assume they have techies. They have a PC shop to take care of the desktop and they hired another consultant for years to maintain the database engine. According to this consultant, it's been very low maintenance considering the lack of investment the have put into the infrastructure. if there is a cloud-based version of Informix they can rent, I should probably move them there.

> I'm guessing they aren't complete morons

No, they are not complete morons. They are lawyers. They treat software and systems the same way they would any appliance or machine. As long as it runs, it's good enough if my memory is correct, I think a couple partners have taken the Mercedes-Benz diesel's well over 200,000 miles they probably won't get rid of them until they are old.

While we know computer systems i.e. software and hardware, are the tech equivalent of fast fashion (i.e. cheaply made and easily ruined), others look at the cost and figure it should last them many many years. I'm probably part of that camp since my laptop is a Dell e6400.

I want to thank you for giving me a solution that wasn't what I expected but, infinitely more useful. Which is that if they don't understand and agree to the investment needed to move to the future, this is one job I should walk away from.

#6

Hi :slight_smile:
Err, i think that was Paul's point!

That and further posts made me look-up Redhat version numbers in
DistroWatch;
http://distrowatch.com/table.php?distribution=redhat
The last release date of 5.2, "apollo", was October 1998 so there is a lot
of wriggle room there.

Walking away sounds difficult but giving them a list of things that needs
to be done might give you some room to manoeuvre and give you some
disclaimers in advance.

Linus offers upgrade routes that Windows Server couldn't hope to offer. It
might even be possible to do most or even all of it remotely or very
cheaply or surprisingly quickly without down-time and with an ever-present
option to revert to the old Redhat version.

It's even possible to upgrade to CentOS as it's pretty much identical to
Redhat but only lacks their technical support = another option Windows
Server doesn't offer!

One route i quite fancy would be to;
1. get a fairly low level techie to plug in an extra ancient hard-drive
(if they can find one that has the ancient connectors or use an adapter).
2. Get their consultant to install CentOS. Reboot once to get into CentOS
and test it. Reboot back into Redhat to solve teething troubles.

It might even be possible to do step 2 remotely!

Their consultant might prefer to dodge both those steps and just get a
slightly less ancient machine and install CentOS on that before posting it
to them. Maybe just doing the finishing touches to tailor a generic CentOS
machine

Moving to an Ubuntu Server might make some sense and even such a different
family (Debian family rather than Redhat family) only has a few trivial
(mostly) differences. Probably is best to stay in the same family though
because it's more familiar to the consultant (or is it? Ubuntu has become
very popular).

The consultant would probably be able to tell which way they would prefer,
and therefore which is likely to be the cheapest route.

Regards from
Tom :slight_smile: