Web lists-archives.com

Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ?




On Mon, 2017-03-27 at 11:11 +0200, Gaetan SLONGO via samba wrote:
> Thank you for your message Andrew. Then .. Waiting for this
> improvement... :-) 
> 
> Indexes seem correct to me. But I'm not sure at 100% because I cannot
> find clear explanation regarding the searchFlags attribute value ..
> 1, 2, ... ? Maybe you have one ? 

https://msdn.microsoft.com/en-us/library/ms679765(v=vs.85).aspx

> Thanks ! 
> 
> ----- Mail original -----
> 
> De: "Andrew Bartlett" <abartlet@xxxxxxxxx> 
> À: "Gaetan SLONGO" <gslongo@xxxxxxxxxxxxx>, "L.P.H. van Belle" <belle
> @bazuin.nl> 
> Cc: samba@xxxxxxxxxxxxxxx 
> Envoyé: Lundi 27 Mars 2017 11:06:56 
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ? 
> 
> On Mon, 2017-03-27 at 10:43 +0200, Gaetan SLONGO via samba wrote: 
> > Zarafa is not on the same server as Samba 
> > 
> > We only have 2 AD/DC Samba 4.5 (CentOS 7) and we put required
> > indexes 
> > on LDAP . 
> > 
> > Arround 1000 mailboxes but not all are simultaneously in use
> > (approx 
> > 1/3 in use). 
> > MTA is postfix (and is still connected to Samba AD, this one is
> > not 
> > causing the issue). 
> > As a workarround, we currently deployed a synchronization
> > connector 
> > from AD to OpenLDAP. It solves the performance issue during the 
> > investigation because Zarafa was totally unusable at all when 
> > connected to Samba... But We plan to move to Zimbra by the end of
> > the 
> > year so I think the work arround can stay in place until the 
> > migration. However this performance issue could be a bottleneck in 
> > other applications, in the future... 
> > 
> > I did not found any config setting allowing tu enable multi-
> > threading 
> > on Samba LDAP backend (maybe an hidden one ?).. I think it could
> > help 
> > a lot 
> 
> Given these discussions, I'm keen to add it. I was going to add this 
> for Samba 4.6.0, but the initial approach I used was slower in some 
> cases (the connect/bind/disconnect case). Sadly at the time there 
> wasn't this level of concern regarding the LDAP performance, so we 
> focussed on what we could achieve, which was making NETLOGON multi- 
> process. 
> 
> This remains on my radar, along with any other approaches we find
> along 
> the way to make search-heavy operation practical. 
> 
> I'm sorry this is causing so much trouble, and I look forward to 
> helping improve this area. 
> 
> In the meantime, adding the indexes that your client tools need will 
> help a lot. 
> 
> Andrew Bartlett 
> 
> > ----- Mail original ----- 
> > 
> > De: "L.P.H. van Belle via samba" <samba@xxxxxxxxxxxxxxx> 
> > À: samba@xxxxxxxxxxxxxxx 
> > Envoyé: Lundi 27 Mars 2017 10:26:22 
> > Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> > unusable), performance tunning ? 
> > 
> > Can you tell more about your setup? 
> > 
> > Is zarafa and samba on the same server for example. 
> > 
> > Which MTA are you using postfix/exim? 
> > 
> > 
> > 
> > My top was about 150 users, and all my printers are connected also
> > so 
> > about 200 devices do ldap searches. 
> > 
> > but my setup is split over 10+ servers ( 2 are AD DC ) 
> > 
> > 
> > 
> > So best is to tell what you can about your setup, anonimize if 
> > needed. 
> > 
> > 
> > 
> > Greetz, 
> > 
> > 
> > 
> > Louis 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > Van: Gaetan SLONGO [mailto:gslongo@xxxxxxxxxxxxx] ;
> > Verzonden: maandag 27 maart 2017 10:12 
> > Aan: L.P.H. van Belle 
> > CC: samba@xxxxxxxxxxxxxxx 
> > Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> > unusable), performance tunning ? 
> > 
> > 
> > 
> > 
> > What we found is Zarafa makes a very big amount of queries, which 
> > makes Samba run at 100% CPU (one process, LDAP does not seems to
> > be 
> > multi-threaded..?)... but we have hundreds of users... 
> > 
> > What do you think could be wrong in the current database/setup ?
> > We 
> > verified all the setup and everything seems OK 
> > 
> > 
> > De: "L.P.H. van Belle via samba" <samba@xxxxxxxxxxxxxxx> 
> > À: samba@xxxxxxxxxxxxxxx 
> > Envoyé: Lundi 27 Mars 2017 09:58:55 
> > Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> > unusable), performance tunning ? 
> > 
> > No, you have to do that manualy, or look the the samba4 ADS script 
> > for kopano ( or zarafa ) 
> > 
> > But I mostly follow the documentation. 
> > 
> > 
> > 
> > And when i run : 
> > 
> > time ldbsearch -H /var/lib/samba/private/sam.ldb -s base -b 
> > @INDEXLIST 
> > 
> > .... 
> > 
> > real 0m0.230s 
> > 
> > user 0m0.184s 
> > 
> > sys 0m0.044s 
> > 
> > 
> > 
> > so if yours take more that 20 sec there is something very wrong. 
> > 
> > I suggest check you samba AD database and samba4 ADDC setup, 
> > 
> > i dont think this is zarafa related. 
> > 
> > 
> > 
> > 
> > 
> > Greetz, 
> > 
> > 
> > 
> > Louis 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > Van: Gaetan SLONGO [mailto:gslongo@xxxxxxxxxxxxx] ;
> > Verzonden: maandag 27 maart 2017 8:46 
> > Aan: L.P.H. van Belle 
> > CC: samba@xxxxxxxxxxxxxxx 
> > Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> > unusable), performance tunning ? 
> > 
> > 
> > 
> > 
> > Hi ! 
> > 
> > Thanks for answer. Yes we use zarafaAccount in search filter. 
> > There is an installer provided for Samba4 to install new schemas ? 
> > 
> > Thanks ! 
> > 
> > 
> > De: "L.P.H. van Belle via samba" <samba@xxxxxxxxxxxxxxx> 
> > À: samba@xxxxxxxxxxxxxxx 
> > Envoyé: Jeudi 23 Mars 2017 11:54:50 
> > Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> > unusable), performance tunning ? 
> > 
> > 
> > Are use using zarafaAccount=1 withing the search filters? 
> > I use this things like this : 
> > 
> > (&(objectClass=person)(zarafaAccount=1)(|(mail=%s)(otherMailbox=%s)
> > )) 
> > 
> > Or for groups. 
> > (&(objectclass=group)(zarafaAccount=1)(|(mail=%s)(otherMailbox=%s))
> > ) 
> > 
> > That helps a lot. 
> > 
> > ! If you switch to kopano beware to change the SCHEMA and filters 
> > zarafaAccount changed to kopanoAccount 
> > 
> > 
> > Greetz. 
> > 
> > Louis 
> > 
> > 
> > > -----Oorspronkelijk bericht----- 
> > > Van: samba [mailto:samba-bounces@xxxxxxxxxxxxxxx] Namens Gaetan 
> > > SLONGO via 
> > > samba 
> > > Verzonden: donderdag 23 maart 2017 11:12 
> > > Aan: samba@xxxxxxxxxxxxxxx 
> > > Onderwerp: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> > > unusable), 
> > > performance tunning ? 
> > > Urgentie: Hoog 
> > > 
> > > 
> > > Dear users, 
> > > 
> > > We are facing to a big latency issue regarding the LDAP Server 
> > > (both 
> > > encrypted & plain). 
> > > 
> > > We have a Zarafa mail server which makes a lot of queries and
> > > puts 
> > > a samba 
> > > process to 100% usage. This latency makes the mail server 
> > > unusable.. The 
> > > mail server was previously on OpenLDAP and there was not 
> > > performance 
> > > issues. 
> > > 
> > > A simple LDAP query can take up to 25 sec to perform !! 
> > > 
> > > We have added some indexes : 
> > > 
> > > [root@califix ~]# ldbsearch -H /var/lib/samba/private/sam.ldb -s 
> > > base -b 
> > > @INDEXLIST 
> > > # record 1 
> > > dn: @INDEXLIST 
> > > @IDXONE: 1 
> > > @IDXVERSION: 2 
> > > @IDXATTR: objectClass 
> > > @IDXATTR: msDS-Cached-Membership-Time-Stamp 
> > > @IDXATTR: userPrincipalName 
> > > @IDXATTR: rpcNsInterfaceID 
> > > @IDXATTR: fileExtPriority 
> > > @IDXATTR: dnsRoot 
> > > @IDXATTR: mSMQLabelEx 
> > > @IDXATTR: dNSTombstoned 
> > > @IDXATTR: msDS-PhoneticCompanyName 
> > > @IDXATTR: msSFU30Domains 
> > > @IDXATTR: dhcpType 
> > > @IDXATTR: ou 
> > > @IDXATTR: gidNumber 
> > > @IDXATTR: msFVE-VolumeGuid 
> > > @IDXATTR: msTSManagingLS2 
> > > @IDXATTR: implementedCategories 
> > > @IDXATTR: oMTIndxGuid 
> > > @IDXATTR: cOMClassID 
> > > @IDXATTR: volTableIdxGUID 
> > > @IDXATTR: l 
> > > @IDXATTR: mSMQDigests 
> > > @IDXATTR: msTSExpireDate4 
> > > @IDXATTR: flatName 
> > > @IDXATTR: msSFU30YpServers 
> > > @IDXATTR: packageFlags 
> > > @IDXATTR: mSMQOwnerID 
> > > @IDXATTR: objectCategory 
> > > @IDXATTR: msSFU30IsValidContainer 
> > > @IDXATTR: msTSProperty02 
> > > @IDXATTR: mS-DS-CreatorSID 
> > > @IDXATTR: proxyAddresses 
> > > @IDXATTR: msPKI-Cert-Template-OID 
> > > @IDXATTR: uNCName 
> > > @IDXATTR: mS-SQL-Name 
> > > @IDXATTR: fSMORoleOwner 
> > > @IDXATTR: msSFU30NisDomain 
> > > @IDXATTR: otherMailbox 
> > > @IDXATTR: location 
> > > @IDXATTR: msSFU30NetgroupHostAtDomain 
> > > @IDXATTR: uSNChanged 
> > > @IDXATTR: sIDHistory 
> > > @IDXATTR: birthLocation 
> > > @IDXATTR: msDS-SecondaryKrbTgtNumber 
> > > @IDXATTR: msTSProperty01 
> > > @IDXATTR: msTSManagingLS4 
> > > @IDXATTR: msSFU30OrderNumber 
> > > @IDXATTR: msDS-HABSeniorityIndex 
> > > @IDXATTR: primaryGroupID 
> > > @IDXATTR: mSMQQueueType 
> > > @IDXATTR: msDFSR-ReplicationGroupGuid 
> > > @IDXATTR: msDS-PhoneticDepartment 
> > > @IDXATTR: mail 
> > > @IDXATTR: msSFU30Name 
> > > @IDXATTR: msSFU30NetgroupUserAtDomain 
> > > @IDXATTR: fromServer 
> > > @IDXATTR: displayName 
> > > @IDXATTR: msTSLicenseVersion2 
> > > @IDXATTR: groupType 
> > > @IDXATTR: msTSLicenseVersion3 
> > > @IDXATTR: msTSLicenseVersion4 
> > > @IDXATTR: userAccountControl 
> > > @IDXATTR: physicalLocationObject 
> > > @IDXATTR: servicePrincipalName 
> > > @IDXATTR: msTSExpireDate 
> > > @IDXATTR: serviceClassName 
> > > @IDXATTR: lDAPDisplayName 
> > > @IDXATTR: zarafaAccount 
> > > @IDXATTR: terminalServer 
> > > @IDXATTR: givenName 
> > > @IDXATTR: msTSManagingLS3 
> > > @IDXATTR: msSFU30MaxUidNumber 
> > > @IDXATTR: msDS-Entry-Time-To-Die 
> > > @IDXATTR: msTSLSProperty01 
> > > @IDXATTR: msDS-PhoneticFirstName 
> > > @IDXATTR: trustPartner 
> > > @IDXATTR: msTSLSProperty02 
> > > @IDXATTR: msTSExpireDate3 
> > > @IDXATTR: objectGUID 
> > > @IDXATTR: showInAdvancedViewOnly 
> > > @IDXATTR: rpcNsTransferSyntax 
> > > @IDXATTR: sAMAccountName 
> > > @IDXATTR: mS-SQL-Version 
> > > @IDXATTR: msDS-Site-Affinity 
> > > @IDXATTR: sn 
> > > @IDXATTR: name 
> > > @IDXATTR: nETBIOSName 
> > > @IDXATTR: sAMAccountType 
> > > @IDXATTR: msTSManagingLS 
> > > @IDXATTR: msDFSR-DfsPath 
> > > @IDXATTR: altSecurityIdentities 
> > > @IDXATTR: USNIntersite 
> > > @IDXATTR: msSFU30MasterServerName 
> > > @IDXATTR: msDS-PhoneticLastName 
> > > @IDXATTR: cn 
> > > @IDXATTR: netbootGUID 
> > > @IDXATTR: lastLogonTimestamp 
> > > @IDXATTR: legacyExchangeDN 
> > > @IDXATTR: mSMQLabel 
> > > @IDXATTR: uSNCreated 
> > > @IDXATTR: mS-SQL-Database 
> > > @IDXATTR: msDS-PhoneticDisplayName 
> > > @IDXATTR: msSFU30MaxGidNumber 
> > > @IDXATTR: rpcNsObjectID 
> > > @IDXATTR: timeVolChange 
> > > @IDXATTR: msTSExpireDate2 
> > > @IDXATTR: groupAttributes 
> > > @IDXATTR: physicalDeliveryOfficeName 
> > > @IDXATTR: msFVE-RecoveryGuid 
> > > @IDXATTR: msDS-AdditionalSamAccountName 
> > > @IDXATTR: objectSid 
> > > @IDXATTR: keywords 
> > > @IDXATTR: mS-SQL-Alias 
> > > @IDXATTR: invocationId 
> > > @IDXATTR: msTSLicenseVersion 
> > > @IDXATTR: requiredCategories 
> > > @IDXATTR: msDS-AzObjectGuid 
> > > distinguishedName: @INDEXLIST 
> > > 
> > > There is any way to improve LDAP responses times ? It seems
> > > there 
> > > is only 
> > > one process which is managing LDAP queries (no forks/threads?) 
> > > 
> > > Thank you in advance for your help !! 
> > > 
> > > -- 
> > > To unsubscribe from this list go to the following URL and read
> > > the 
> > > instructions: https://lists.samba.org/mailman/options/samba ;
> > 
> > 
> > 
> > -- 
> > To unsubscribe from this list go to the following URL and read the 
> > instructions: https://lists.samba.org/mailman/options/samba ;
> > 
> > 
> > 
> > 
> > 
> > -- 
> > 
> > 
> > 
> > www.it-optics.com 
> > 
> > Gaëtan SLONGO | Head of Infrastructure Department 
> > Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> > 
> > Company : 
> > 
> > +32 (0)65 84 23 85 
> > 
> > Direct : 
> > 
> > +32 (0)65 32 85 88 
> > 
> > Fax : 
> > 
> > +32 (0)65 84 66 76 
> > 
> > Skype ID : 
> > 
> > gslongo.pro 
> > 
> > GPG Key : 
> > 
> > gslongo-gpg_key.asc 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > - Please consider your environmental responsibility before
> > printing 
> > this e-mail - 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > -- 
> > To unsubscribe from this list go to the following URL and read the 
> > instructions: https://lists.samba.org/mailman/options/samba ;
> > 
> > 
> > 
> > 
> > 
> > -- 
> > 
> > 
> > 
> > www.it-optics.com 
> > 
> > Gaëtan SLONGO | Head of Infrastructure Department 
> > Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> > 
> > Company : 
> > 
> > +32 (0)65 84 23 85 
> > 
> > Direct : 
> > 
> > +32 (0)65 32 85 88 
> > 
> > Fax : 
> > 
> > +32 (0)65 84 66 76 
> > 
> > Skype ID : 
> > 
> > gslongo.pro 
> > 
> > GPG Key : 
> > 
> > gslongo-gpg_key.asc 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > - Please consider your environmental responsibility before
> > printing 
> > this e-mail - 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > -- 
> > To unsubscribe from this list go to the following URL and read the 
> > instructions: https://lists.samba.org/mailman/options/samba ;
> > 
> > 
> > 
> > -- 
> > 
> > 
> > 
> > 
> > www.it-optics.com 
> > 
> > Gaëtan SLONGO | Head of Infrastructure Department 
> > Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> > Company : +32 (0)65 84 23 85 
> > Direct : +32 (0)65 32 85 88 
> > Fax : +32 (0)65 84 66 76 
> > Skype ID : gslongo.pro 
> > GPG Key : gslongo-gpg_key.asc 
> > 
> > 
> > - Please consider your environmental responsibility before
> > printing 
> > this e-mail - 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> 
> -- 
> Andrew Bartlett http://samba.org/~abartlet/ ;
> Authentication Developer, Samba Team http://samba.org ;
> Samba Developer, Catalyst IT http://catalyst.net.nz/services/samba ;
> 
> 
> 
> 
> -- 
> 
> 
> 
> 
> www.it-optics.com 
> 	
> Gaëtan SLONGO | Head of Infrastructure Department 
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> Company : 	+32 (0)65 84 23 85 
> Direct : 	+32 (0)65 32 85 88 
> Fax : 	+32 (0)65 84 66 76 
> Skype ID : 	gslongo.pro 
> GPG Key : 	gslongo-gpg_key.asc 
> 	
> 
> - Please consider your environmental responsibility before printing
> this e-mail - 
> 
> 
> 
> 
> 
> 
> 
> 
-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba


-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba