spamassassin-users January 2011 archive
Main Archive Page > Month Archives  > spamassassin-users archives
spamassassin-users: Re: sa-learn --force-expire hangs [SOLVED]

Re: sa-learn --force-expire hangs [SOLVED]

From: John Adams <mailinglists_at_nospam>
Date: Fri Jan 28 2011 - 14:56:10 GMT

Am 28.01.2011 15:25, schrieb John Hardin:
> On Fri, 28 Jan 2011, John Adams wrote:
>> Problem solved.
> For the benefit of others searching the archives in the future, how?

the reason for the hanging was the mysql db. Top permanently showed an
iowait load of ~50% on a quad core machine. Running sa-learn
--force-expire caused the SQL query in mysqld to hang. KILL [process] in
mysqld's CLI dit not resolve anything. kill -9 [process_id of mysqld]
was the only way how to resolve the hanging. This possibly corrupted the
bayes_* tables that were using innodb. Killing mysqld let iowait drop to 0.

the solution was to sa-learn --backup the bayes_* tables from the old
database, drop the old bayes_* tables, and then recreate them in a new
DB. The sa-learn --backup procedure took a few hours.
I used sa-learn --restore to put back all 8m tokens into the mysql db
(this time using the myisam engine). The restore took about 24 hours.
After that, I ran sa-learn --force-expire which deleted 7.9m old tokens
in 225 seconds.

I suspect either an incompatibility between the DB scheme from sa 3.2.x
and 3.3.x (3.3.1 complained about the DB version and refused to work
when I recreated the tables using the old scheme I used for sa 3.2.x),
and/or the old innodb bayes_* tables already were corrupt from even
older DB crashes.

Please share your thoughts on this. Would there have been better ways?