Web lists-archives.com

Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ?




I reviewed you indexing list again against mine. 

 

A diff between your indexing list and mine shows just a small difference.

 

> @IDXATTR: otherMailbox

109d109

< @IDXATTR: uidNumber

116a117

> @IDXATTR: zarafaAccount

 

Left (<) me, right (>) yours.

 

You did you install the zarafa schema in the AD? 

I did use : zarafa_schema_add.sh 

 

Beside that i dont see much. 

 

 

Greetz, 

 

Louis

 

 

 

 


Van: Gaetan SLONGO [mailto:gslongo@xxxxxxxxxxxxx] 
Verzonden: maandag 27 maart 2017 11:12
Aan: Andrew Bartlett
CC: samba@xxxxxxxxxxxxxxx; L.P.H. van Belle
Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ?


 

Thank you for your message Andrew. Then .. Waiting for this improvement... :-)

Indexes seem correct to me. But I'm not sure at 100% because I cannot find clear explanation regarding the searchFlags attribute value .. 1, 2, ... ? Maybe you have one ?

Thanks !


De: "Andrew Bartlett" <abartlet@xxxxxxxxx>
À: "Gaetan SLONGO" <gslongo@xxxxxxxxxxxxx>, "L.P.H. van Belle" <belle@xxxxxxxxx>
Cc: samba@xxxxxxxxxxxxxxx
Envoyé: Lundi 27 Mars 2017 11:06:56
Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ?

On Mon, 2017-03-27 at 10:43 +0200, Gaetan SLONGO via samba wrote:
> Zarafa is not on the same server as Samba 
> 
> We only have 2 AD/DC Samba 4.5 (CentOS 7) and we put required indexes
> on LDAP . 
> 
> Arround 1000 mailboxes but not all are simultaneously in use (approx
> 1/3 in use). 
> MTA is postfix (and is still connected to Samba AD, this one is not
> causing the issue). 
> As a workarround, we currently deployed a synchronization connector
> from AD to OpenLDAP. It solves the performance issue during the
> investigation because Zarafa was totally unusable at all when
> connected to Samba... But We plan to move to Zimbra by the end of the
> year so I think the work arround can stay in place until the
> migration. However this performance issue could be a bottleneck in
> other applications, in the future... 
> 
> I did not found any config setting allowing tu enable multi-threading 
> on Samba LDAP backend (maybe an hidden one ?).. I think it could help
> a lot 

Given these discussions, I'm keen to add it.  I was going to add this
for Samba 4.6.0, but the initial approach I used was slower in some
cases (the connect/bind/disconnect case).  Sadly at the time there
wasn't this level of concern regarding the LDAP performance, so we
focussed on what we could achieve, which was making NETLOGON multi-
process.

This remains on my radar, along with any other approaches we find along
the way to make search-heavy operation practical. 

I'm sorry this is causing so much trouble, and I look forward to
helping improve this area.  

In the meantime, adding the indexes that your client tools need will
help a lot.

Andrew Bartlett

> ----- Mail original -----
> 
> De: "L.P.H. van Belle via samba" <samba@xxxxxxxxxxxxxxx> 
> À: samba@xxxxxxxxxxxxxxx 
> Envoyé: Lundi 27 Mars 2017 10:26:22 
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ? 
> 
> Can you tell more about your setup? 
> 
> Is zarafa and samba on the same server for example. 
> 
> Which MTA are you using postfix/exim? 
> 
> 
> 
> My top was about 150 users, and all my printers are connected also so
> about 200 devices do ldap searches. 
> 
> but my setup is split over 10+ servers ( 2 are AD DC ) 
> 
> 
> 
> So best is to tell what you can about your setup, anonimize if
> needed. 
> 
> 
> 
> Greetz, 
> 
> 
> 
> Louis 
> 
> 
> 
> 
> 
> 
> 
> 
> Van: Gaetan SLONGO [mailto:gslongo@xxxxxxxxxxxxx] ;
> Verzonden: maandag 27 maart 2017 10:12 
> Aan: L.P.H. van Belle 
> CC: samba@xxxxxxxxxxxxxxx 
> Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ? 
> 
> 
> 
> 
> What we found is Zarafa makes a very big amount of queries, which
> makes Samba run at 100% CPU (one process, LDAP does not seems to be
> multi-threaded..?)... but we have hundreds of users... 
> 
> What do you think could be wrong in the current database/setup ? We
> verified all the setup and everything seems OK 
> 
> 
> De: "L.P.H. van Belle via samba" <samba@xxxxxxxxxxxxxxx> 
> À: samba@xxxxxxxxxxxxxxx 
> Envoyé: Lundi 27 Mars 2017 09:58:55 
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ? 
> 
> No, you have to do that manualy, or look the the samba4 ADS script
> for kopano ( or zarafa ) 
> 
> But I mostly follow the documentation. 
> 
> 
> 
> And when i run : 
> 
> time ldbsearch -H /var/lib/samba/private/sam.ldb -s base -b
> @INDEXLIST 
> 
> .... 
> 
> real 0m0.230s 
> 
> user 0m0.184s 
> 
> sys 0m0.044s 
> 
> 
> 
> so if yours take more that 20 sec there is something very wrong. 
> 
> I suggest check you samba AD database and samba4 ADDC setup, 
> 
> i dont think this is zarafa related. 
> 
> 
> 
> 
> 
> Greetz, 
> 
> 
> 
> Louis 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Van: Gaetan SLONGO [mailto:gslongo@xxxxxxxxxxxxx] ;
> Verzonden: maandag 27 maart 2017 8:46 
> Aan: L.P.H. van Belle 
> CC: samba@xxxxxxxxxxxxxxx 
> Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ? 
> 
> 
> 
> 
> Hi ! 
> 
> Thanks for answer. Yes we use zarafaAccount in search filter. 
> There is an installer provided for Samba4 to install new schemas ? 
> 
> Thanks ! 
> 
> 
> De: "L.P.H. van Belle via samba" <samba@xxxxxxxxxxxxxxx> 
> À: samba@xxxxxxxxxxxxxxx 
> Envoyé: Jeudi 23 Mars 2017 11:54:50 
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ? 
> 
> 
> Are use using zarafaAccount=1 withing the search filters? 
> I use this things like this : 
> 
> (&(objectClass=person)(zarafaAccount=1)(|(mail=%s)(otherMailbox=%s)))
>  
> Or for groups. 
> (&(objectclass=group)(zarafaAccount=1)(|(mail=%s)(otherMailbox=%s))) 
> 
> That helps a lot. 
> 
> ! If you switch to kopano beware to change the SCHEMA and filters 
> zarafaAccount changed to kopanoAccount 
> 
> 
> Greetz. 
> 
> Louis 
> 
> 
> > -----Oorspronkelijk bericht----- 
> > Van: samba [mailto:samba-bounces@xxxxxxxxxxxxxxx] Namens Gaetan
> > SLONGO via 
> > samba 
> > Verzonden: donderdag 23 maart 2017 11:12 
> > Aan: samba@xxxxxxxxxxxxxxx 
> > Onderwerp: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> > unusable), 
> > performance tunning ? 
> > Urgentie: Hoog 
> > 
> > 
> > Dear users, 
> > 
> > We are facing to a big latency issue regarding the LDAP Server
> > (both 
> > encrypted & plain). 
> > 
> > We have a Zarafa mail server which makes a lot of queries and puts
> > a samba 
> > process to 100% usage. This latency makes the mail server
> > unusable.. The 
> > mail server was previously on OpenLDAP and there was not
> > performance 
> > issues. 
> > 
> > A simple LDAP query can take up to 25 sec to perform !! 
> > 
> > We have added some indexes : 
> > 
> > [root@califix ~]# ldbsearch -H /var/lib/samba/private/sam.ldb -s
> > base -b 
> > @INDEXLIST 
> > # record 1 
> > dn: @INDEXLIST 
> > @IDXONE: 1 
> > @IDXVERSION: 2 
> > @IDXATTR: objectClass 
> > @IDXATTR: msDS-Cached-Membership-Time-Stamp 
> > @IDXATTR: userPrincipalName 
> > @IDXATTR: rpcNsInterfaceID 
> > @IDXATTR: fileExtPriority 
> > @IDXATTR: dnsRoot 
> > @IDXATTR: mSMQLabelEx 
> > @IDXATTR: dNSTombstoned 
> > @IDXATTR: msDS-PhoneticCompanyName 
> > @IDXATTR: msSFU30Domains 
> > @IDXATTR: dhcpType 
> > @IDXATTR: ou 
> > @IDXATTR: gidNumber 
> > @IDXATTR: msFVE-VolumeGuid 
> > @IDXATTR: msTSManagingLS2 
> > @IDXATTR: implementedCategories 
> > @IDXATTR: oMTIndxGuid 
> > @IDXATTR: cOMClassID 
> > @IDXATTR: volTableIdxGUID 
> > @IDXATTR: l 
> > @IDXATTR: mSMQDigests 
> > @IDXATTR: msTSExpireDate4 
> > @IDXATTR: flatName 
> > @IDXATTR: msSFU30YpServers 
> > @IDXATTR: packageFlags 
> > @IDXATTR: mSMQOwnerID 
> > @IDXATTR: objectCategory 
> > @IDXATTR: msSFU30IsValidContainer 
> > @IDXATTR: msTSProperty02 
> > @IDXATTR: mS-DS-CreatorSID 
> > @IDXATTR: proxyAddresses 
> > @IDXATTR: msPKI-Cert-Template-OID 
> > @IDXATTR: uNCName 
> > @IDXATTR: mS-SQL-Name 
> > @IDXATTR: fSMORoleOwner 
> > @IDXATTR: msSFU30NisDomain 
> > @IDXATTR: otherMailbox 
> > @IDXATTR: location 
> > @IDXATTR: msSFU30NetgroupHostAtDomain 
> > @IDXATTR: uSNChanged 
> > @IDXATTR: sIDHistory 
> > @IDXATTR: birthLocation 
> > @IDXATTR: msDS-SecondaryKrbTgtNumber 
> > @IDXATTR: msTSProperty01 
> > @IDXATTR: msTSManagingLS4 
> > @IDXATTR: msSFU30OrderNumber 
> > @IDXATTR: msDS-HABSeniorityIndex 
> > @IDXATTR: primaryGroupID 
> > @IDXATTR: mSMQQueueType 
> > @IDXATTR: msDFSR-ReplicationGroupGuid 
> > @IDXATTR: msDS-PhoneticDepartment 
> > @IDXATTR: mail 
> > @IDXATTR: msSFU30Name 
> > @IDXATTR: msSFU30NetgroupUserAtDomain 
> > @IDXATTR: fromServer 
> > @IDXATTR: displayName 
> > @IDXATTR: msTSLicenseVersion2 
> > @IDXATTR: groupType 
> > @IDXATTR: msTSLicenseVersion3 
> > @IDXATTR: msTSLicenseVersion4 
> > @IDXATTR: userAccountControl 
> > @IDXATTR: physicalLocationObject 
> > @IDXATTR: servicePrincipalName 
> > @IDXATTR: msTSExpireDate 
> > @IDXATTR: serviceClassName 
> > @IDXATTR: lDAPDisplayName 
> > @IDXATTR: zarafaAccount 
> > @IDXATTR: terminalServer 
> > @IDXATTR: givenName 
> > @IDXATTR: msTSManagingLS3 
> > @IDXATTR: msSFU30MaxUidNumber 
> > @IDXATTR: msDS-Entry-Time-To-Die 
> > @IDXATTR: msTSLSProperty01 
> > @IDXATTR: msDS-PhoneticFirstName 
> > @IDXATTR: trustPartner 
> > @IDXATTR: msTSLSProperty02 
> > @IDXATTR: msTSExpireDate3 
> > @IDXATTR: objectGUID 
> > @IDXATTR: showInAdvancedViewOnly 
> > @IDXATTR: rpcNsTransferSyntax 
> > @IDXATTR: sAMAccountName 
> > @IDXATTR: mS-SQL-Version 
> > @IDXATTR: msDS-Site-Affinity 
> > @IDXATTR: sn 
> > @IDXATTR: name 
> > @IDXATTR: nETBIOSName 
> > @IDXATTR: sAMAccountType 
> > @IDXATTR: msTSManagingLS 
> > @IDXATTR: msDFSR-DfsPath 
> > @IDXATTR: altSecurityIdentities 
> > @IDXATTR: USNIntersite 
> > @IDXATTR: msSFU30MasterServerName 
> > @IDXATTR: msDS-PhoneticLastName 
> > @IDXATTR: cn 
> > @IDXATTR: netbootGUID 
> > @IDXATTR: lastLogonTimestamp 
> > @IDXATTR: legacyExchangeDN 
> > @IDXATTR: mSMQLabel 
> > @IDXATTR: uSNCreated 
> > @IDXATTR: mS-SQL-Database 
> > @IDXATTR: msDS-PhoneticDisplayName 
> > @IDXATTR: msSFU30MaxGidNumber 
> > @IDXATTR: rpcNsObjectID 
> > @IDXATTR: timeVolChange 
> > @IDXATTR: msTSExpireDate2 
> > @IDXATTR: groupAttributes 
> > @IDXATTR: physicalDeliveryOfficeName 
> > @IDXATTR: msFVE-RecoveryGuid 
> > @IDXATTR: msDS-AdditionalSamAccountName 
> > @IDXATTR: objectSid 
> > @IDXATTR: keywords 
> > @IDXATTR: mS-SQL-Alias 
> > @IDXATTR: invocationId 
> > @IDXATTR: msTSLicenseVersion 
> > @IDXATTR: requiredCategories 
> > @IDXATTR: msDS-AzObjectGuid 
> > distinguishedName: @INDEXLIST 
> > 
> > There is any way to improve LDAP responses times ? It seems there
> > is only 
> > one process which is managing LDAP queries (no forks/threads?) 
> > 
> > Thank you in advance for your help !! 
> > 
> > -- 
> > To unsubscribe from this list go to the following URL and read the 
> > instructions: https://lists.samba.org/mailman/options/samba ;
> 
> 
> 
> -- 
> To unsubscribe from this list go to the following URL and read the 
> instructions: https://lists.samba.org/mailman/options/samba ;
> 
> 
> 
> 
> 
> -- 
> 
> 
> 
> www.it-optics.com 
> 
> Gaëtan SLONGO | Head of Infrastructure Department 
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> 
> Company : 
> 
> +32 (0)65 84 23 85 
> 
> Direct : 
> 
> +32 (0)65 32 85 88 
> 
> Fax : 
> 
> +32 (0)65 84 66 76 
> 
> Skype ID : 
> 
> gslongo.pro 
> 
> GPG Key : 
> 
> gslongo-gpg_key.asc 
> 
> 
> 
> 
> 
> 
> 
> - Please consider your environmental responsibility before printing
> this e-mail - 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> To unsubscribe from this list go to the following URL and read the 
> instructions: https://lists.samba.org/mailman/options/samba ;
> 
> 
> 
> 
> 
> -- 
> 
> 
> 
> www.it-optics.com 
> 
> Gaëtan SLONGO | Head of Infrastructure Department 
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> 
> Company : 
> 
> +32 (0)65 84 23 85 
> 
> Direct : 
> 
> +32 (0)65 32 85 88 
> 
> Fax : 
> 
> +32 (0)65 84 66 76 
> 
> Skype ID : 
> 
> gslongo.pro 
> 
> GPG Key : 
> 
> gslongo-gpg_key.asc 
> 
> 
> 
> 
> 
> 
> 
> - Please consider your environmental responsibility before printing
> this e-mail - 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> To unsubscribe from this list go to the following URL and read the 
> instructions: https://lists.samba.org/mailman/options/samba ;
> 
> 
> 
> -- 
> 
> 
> 
> 
> www.it-optics.com 
>         
> Gaëtan SLONGO | Head of Infrastructure Department 
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> Company :         +32 (0)65 84 23 85 
> Direct :         +32 (0)65 32 85 88 
> Fax :         +32 (0)65 84 66 76 
> Skype ID :         gslongo.pro 
> GPG Key :         gslongo-gpg_key.asc 
>         
> 
> - Please consider your environmental responsibility before printing
> this e-mail - 
> 
> 
> 
> 
> 
> 
> 
> 
-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba





-- 



www.it-optics.com 
	
Gaëtan SLONGO | Head of Infrastructure Department
Boulevard Initialis, 28 - 7000 Mons, BELGIUM

Company :
	
+32 (0)65 84 23 85

Direct :
	
+32 (0)65 32 85 88

Fax :
	
+32 (0)65 84 66 76

Skype ID :
	
gslongo.pro

GPG Key :
	
gslongo-gpg_key.asc 



	



- Please consider your environmental responsibility before printing this e-mail -


 


 


 


 


 




-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba