openldap-software May 2010 archive
Main Archive Page > Month Archives  > openldap-software archives
openldap-software: Re: syncrepl_del_nonpresent and syncrepl issu

Re: syncrepl_del_nonpresent and syncrepl issues.

From: Quanah Gibson-Mount <quanah_at_nospam>
Date: Thu May 13 2010 - 15:49:27 GMT
To: Jorgen Lundman <lundman@lundman.net>, openldap-software@openldap.org

--On May 13, 2010 2:23:33 PM +0900 Jorgen Lundman <lundman@lundman.net>
wrote:

>
>
>>>
>>> dbconfig set_lk_detect DB_LOCK_DEFAULT
>>> dbconfig set_lg_max 52428800
>>> dbconfig set_cachesize 0 67108864 1
>>> dbconfig set_flags db_log_autoremove
>>> dbconfig set_lk_max_objects 1500
>>> dbconfig set_lk_max_locks 1500
>>> dbconfig set_lk_max_lockers 1500
>>
>> You don't provide relevant data on the size of your database to make any
>> comment one way or the other on whether your "set_cachesize" value is
>> correct.
>>
>> I would suggest you provide the output of du -c -h *.bdb in your
>> OpenLDAP 2.3 database directory.
>
> Coming right up! I appreciate your time in doing some sanity checks for
> me.
>
> 2.8M DNSData.bdb
> 3.9M DNSHostName.bdb
> 2.4M DNSIPAddr.bdb
> 1.4M DNSType.bdb
> 1.5M accountStatus.bdb
> 258K amavisSpamLover.bdb
> 642K deliveryMode.bdb
> 1.2G dn2id.bdb
> 62M entryCSN.bdb
> 84M entryUUID.bdb
> 13M gecos.bdb
> 898K gidNumber.bdb
> 2.5G id2entry.bdb
> 9.0M mail.bdb
> 7.6M mailAlternateAddress.bdb
> 5.6M o.bdb
> 3.9M objectClass.bdb
> 1.0M radiusGroupName.bdb
> 14M uid.bdb
> 12M uidNumber.bdb
> (no -c in Solaris, the whole dir is: )
> 9.0G .

> Now, live slapd consumes about 800MB of memory when it is running, which
> is probably acceptable as the server is dedicated to LDAP master. (Has
> 2GB RAM).

Your server is not sufficiently powerful enough to optimally run your
database. Your database size is approximately 4GB in size (dn2id.bdb +
id2entry.bdb + index databases). For an optimal slapadd run, your BDB
cachesize (set_cachesize from DB_CONFIG) must be able to contain this
entire dataset in memory. So you have only 1/2 the necessary memory to do
an optimal slapadd load. This is why your slapadd takes 15+ hours to run.
For your database itself to run optimally under slapd, it needs to be able
to hold the sum of dn2id.bdb + id2entry.bdb, or in your case,
1.2G+2.5G=3.7GB. So again, you do not have enough memory. To make matters
worse, your DB_CONFIG only allocates 64MB to slapd. Which, quite frankly,
is an insanely small number for a database the size you are operating.

One other tuning parameter that will help slapadd that I failed to mention
is the tool-threads setting. It should be set to the number of real cores
the box has. However, this setting will not help in your resource
constrained environment.

The first thing you need to do is get new hardware or increase the
available RAM in the system you have. Since the running slapd is going to
take the DB_CACHESIZE + thread overhead + slapd cachesizes memory, you
probably need closer to 8GB of RAM to run your LDAP service.

--Quanah

-- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration