What database?

Want to help out? Need help accessing the AniDB API? This is the place to ask questions.

Moderator: AniDB

Elberet
Posts: 778
Joined: Sat Jul 19, 2003 8:14 pm

What database?

Post by Elberet »

Forgive my nosiness, but what database do you have running in the backend? Thinking about some of the tables, the number of rows and the sizes of the table indexes gave me a real headache, and I'm wondering if MySQL or Postgres can still handle that or if something big (read: expensive) is required...
PetriW
AniDB Staff
Posts: 1522
Joined: Sat May 24, 2003 2:34 pm

Post by PetriW »

Dunno what anidb uses but mysql can handle millions of rows without any problems, mostly ram that matters anyway once you start getting a large database. (If you have optimized queries and indexes of course.)
Problem with mysql without transaction support is that the tables break very easy though.
DonGato
Posts: 1296
Joined: Sun Nov 17, 2002 9:08 pm
Location: The Pampas, The land of the Gaucho!
Contact:

Post by DonGato »

IIRC it is using PostgreSQL.
Elberet
Posts: 778
Joined: Sat Jul 19, 2003 8:14 pm

Post by Elberet »

Nah, MySQL's lack of transaction support doesn't break the tables themselves. The worst thing that could happen is that a foreign key becomes meaningless (i.e. an entry for a file is deleted but the references to that file aren't removed from everyone's mylists). However, that is a problem that can be solved on the application level, and even if that happens, the tables are still not corrupt per se.

Anyhow, after wasting some more thinking time on this, you're probably right. It's unlikely that any given table would contain more then a couple million rows. But now I wonder how much disk space the indices actually consume... ;)
PetriW
AniDB Staff
Posts: 1522
Joined: Sat May 24, 2003 2:34 pm

Post by PetriW »

Actually what I meant is that mysql tables get corrupted pretty easy, those of my friends running larger sites on mysql all have corruption issues now and then. (Not common but also too common to be regarded as safe compared to a transaction database.)
Guess I just wouldn't feel comfortable with a big site running on MyISAM tables. :roll:
exp
Site Admin
Posts: 2438
Joined: Tue Oct 01, 2002 9:42 pm
Location: Nowhere

Post by exp »

AniDB is running on PostgreSQL and (not counting certain cronjobs which run over the entire db now and then) there are no performance issues.
All queries issued by the cgi are handled without any real delay.

About the size, anidb together with all indices takes about 400MB of hdd space atm. And it's growing rapidly :o)

Btw. i think mysql is a nice little proggy to play around with, but if you need a real database (and want an open source one), it's better to stick with postgresql.

BYe!
EXP
sothis
Posts: 28
Joined: Tue Jul 22, 2003 4:15 pm

Post by sothis »

Actually what I meant is that mysql tables get corrupted pretty easy, those of my friends running larger sites on mysql all have corruption issues now and then.
i've encountered this problem myself, probably every 4-5 months or so, with my database (mysql). it randomly will either not be able to access a field, all the data will magically disappear, or something else of the like. after corruption, the database itself couldn't even be deleted. took about a year to discover that in fact, if the mysql server was simply restarted, all the data was back :/ i still have no idea why.
Elberet
Posts: 778
Joined: Sat Jul 19, 2003 8:14 pm

Post by Elberet »

Sounds like your mysqld version leaks memory and either eventually uses up all system memory or runs into a memory limit. But that's just a wild guess... :?
MeqA

Post by MeqA »

so you guys use postgresql to create this site and cron jobs to control the site automaticly?
exp
Site Admin
Posts: 2438
Joined: Tue Oct 01, 2002 9:42 pm
Location: Nowhere

Post by exp »

MeqA wrote:so you guys use postgresql to create this site and cron jobs to control the site automaticly?
yes

btw. status update, db size is now nearly 2 GB :P
Last edited by exp on Sun May 23, 2004 8:56 pm, edited 1 time in total.
DonGato
Posts: 1296
Joined: Sun Nov 17, 2002 9:08 pm
Location: The Pampas, The land of the Gaucho!
Contact:

Post by DonGato »

exp wrote:btw. status update, db size ist now nearly 2 GB :P
Quite small... I thought it would be bigger.
kidan
Posts: 319
Joined: Thu Feb 13, 2003 9:13 pm
Location: .DE

Post by kidan »

hmm, I hope it's highly compressible and you're doing backups regulary...

Getting 2 GB right off the server and burning it to CD daily would be a pain in the a..
ShadowPeo
Posts: 1
Joined: Tue Jun 22, 2004 11:48 pm
Location: Melbourne, Australia
Contact:

Post by ShadowPeo »

MySQL InnoDB does support transactions, I have never used it myself as I only found this out about 2 weeks ago, I am going to use it in my new projects though
MaxVT
Posts: 2
Joined: Sat Dec 18, 2004 4:24 pm
Contact:

Post by MaxVT »

You guys are funny :) MySQL is one of the biggest DB's in the market. In case you didn't know, Apache + PHP + MySQL is a standard server setup for many many sites.

I've been using MySQL for over 4 years now, and for more than 2 years now I didn't hear of any corruption going on (some weird bug wasted a lot of my time a few years ago randomly corrupting one table). Seems like you need to update your version :)
Elias
Posts: 242
Joined: Tue Feb 17, 2004 4:55 pm

Post by Elias »

Biggest doesnt mean better. There are lots of features, which lacks in MySQL (or are implemented not very good or only partially) but are present in PostgresSQL and other DB, like:
- views
- sequences
- constraints
- subqueries
- triggers
- stored procedures
- prepared queries
- cursors
- transactions
Locked