Troubleshooting Tips

This section provides troubleshooting tips for administrators using the IBM Security DirectControl Agent for Mac.

The following topics are covered:

Using Common Account Management Commands

Viewing the Agent Version on the Macs Joined to Active Directory

Enabling Logging for the IBM Security Direct Control Agent for Mac

Enabling Logging for the Mac Directory Service Using the Agent on a Dual-Boot System

Using Adgp Update Appropriately

Understanding Delays when Logging On the First Time with a New User Account

Configuring Single-Sign On to Work with Non-Mac Computers

Restricting Login Using FTP

Logging On Using Local Host

Changing the Password for Active Directory Users

Disabling the Apple Built-In Active Directory Plug-In

Showing the Correct Status of the IBM Security Plug-In

Resolving VPN Access Issues with Mac OS X 10.7 and Later

Diagnosing Smart Card Log In Problems

Opening a Support Case Online Collecting Information for Support Cases

Using Common Account Management Commands

Most UNIX-based platforms store account information in the local /etc/passwd file, and use commands such as getent command to query that information. On Mac computers, however, you would typically use the Directory Service application to manage local accounts and retrieve user information. For troubleshooting purposes, therefore, you should be familiar with the commands to use for retrieving information about Active Directory users and groups.

The following table describes several common Directory Service Command Line (dscl) commands that you may find useful.

Use this command	To do this
`dscl /Search –list /Users`	List all of the users in the Directory Service and in Active Directory for the zone.
`dscl /CentrifyDC –list /Users`	List only the Active Directory users enabled for the zone.
`dscl /CentrifyDC –read /Users/username`	Display detailed information about the specified Active Directory username.
`dscl /Search –list /Groups`	List all of the groups in the Directory Service and in Active Directory for the zone.
`dscl /CentrifyDC –list /Groups`	List only the Active Directory groups enabled for the zone.
`dscl /CentrifyDC –read /Groups/groupname`	Display detailed information about the specified Active Directory groupname.

To get detailed information for all users or groups recognized on the Mac computer, you can use the following commands:

lookupd –q user –a name

lookupd –q group –a name

To get detailed information for a specific user or group, you can use the following commands:

lookupd –q user –a name username

lookupd –q group –a name groupname

To clear the Directory Service cache, you can use the following command:

lookupd -flushcache

To completely clear the cache of Active Directory login credentials, you should also run the `adflush command:

adflush

To retrieve Mac OS version and build information that uname -a does not provide, you can run the following command:

/usr/bin/sw_vers

Viewing the Agent Version on the Macs Joined to Active Directory

You can use the Active Directory module for Windows PowerShell to view the version of the IBM Security DirectControl Agent for Mac on the Macs joined to your AD domain. This is useful to verify that all Macs joined to your AD have an appropriate version of the IBM Security DirectControl Agent for Mac to avoid compatibility issues with OS updates.

Install the Active Directory Module for Windows PowerShell

The Active Directory module for Windows PowerShell is already installed on domain controllers. If you are using a member server, you will have to install it.

To install the Active Directory module for Windows PowerShell

Open an elevated PowerShell session on a Windows server in the domain and run the following command:

Add-WindowsFeature RSAT-AD-PowerShell

When the installation finishes it returns the following:

Once installed, on Windows Server 2012 and 2012 R2 the module automatically loads when you use one of its cmdlets; you do not need to import it.

Show PowerShell Output of Agent Versions for AD-Joined Computers

If you have a small environment, or just want to see a sample of the information that will be in the report, run the following from a Windows server with the AD PowerShell module installed:

Get-ADComputer -Filter * -Properties OperatingSystem,OperatingSystemServicePack | Format-Table Name,OperatingSystem,OperatingSystemServicePack -wrap -auto

For example:

The report includes all AD-joined computers in the domain. The example above shows a mix of Windows, Linux, and Mac computers. Where OperatingSystemServicePack is empty, it means there is no Service Pack installed (Windows computers), or there is no DirectControl agent installed (Mac or Linux/Unix).

In most cases there are too many computers for the PowerShell output to be easily readable. In these cases, refer to the next section, Export the Report of Agent Versions to a CSV File.

Export the Report of Agent Versions to a CSV File

You can export a report of IBM Security DirectControl Agent for Mac versions on AD-joined computers to a CSV file for easier manipulation by running the following:

Get-ADComputer -Filter * -Properties OperatingSystem,OperatingSystemServicePack | Select-Object Name,OperatingSystem,OperatingSystemServicePack | Export-CSV CDCVersion.csv -NoTypeInformation -Encoding UTF8

In this example, PowerShell exports the data described above in Show PowerShell Output of Agent Versions for AD-joined Computers to a CSV file named CDCVersion.csv in the current directory. You can then open that CSV file using a spreadsheet application such as Excel to more easily analyze the data.

Enabling Logging for the IBM Security DirectControl Agent for Mac

The IBM Security DirectControl Agent for Mac installation includes some basic diagnostic tools and a logging mechanism to help you trace the source of problems if they occur. These diagnostic tools and log files allow you to periodically check your environment and view information about the agent operation, your Active Directory connections, and the configuration settings for individual computers.

In most cases, logging is not enabled by default for performance reasons. Once enabled, however, log files provide a detailed record of IBM Security DirectControl Agent for Mac activity and can be used to analyze the behavior of IBM Security Management Services and communication with Active Directory to locate points of failure.

For performance and security reasons, you should only enable agent logging when necessary, for example, when requested to do so by IBM Security Corporation Technical Support, and for short periods of time to diagnose a problem. Keep in mind that sensitive information may be written to this file and you should evaluate the contents of the file before giving others access to it.

You can enable logging either by using the cdcdebug command or the IBM Security for Mac Diagnostic Tool application.

To enable logging with the cdcdebug command:

Log in to the Mac as Local Admin and open the Terminal.
Run the following commands to clear and then enable the IBM Security DirectControl Agent for Mac log file:

% sudo /usr/local/share/centrifydc/bin/cdcdebug clear

% sudo /usr/local/share/centrifydc/bin/cdcdebug on
Record the start time point:

% date +%s

For example: the output is 1610614011, please remember this output, it is the start time point.
Log out of the local admin user account.
Reproduce the issue: try to log in as the affected Active Directory user. Let it fail.
Log back in as Local Admin and open the Terminal again.
Record the end time point:

% date +%s

For example: the output is 1610614043, please remember this output, it is the end time point)
Enter the following commands to collect the IBM Security DirectControl Agent for Mac log file:

% sudo /usr/local/share/centrifydc/bin/cdcdebug -f pack [affected_AD_user_name] [start_time_point] [end_time_point]

% adquery user -A [affected_AD_user_name] > /tmp/adquery.log
Send us the following files for analysis:

/var/centrify/tmp/cdcdebug.tar.gz

/tmp/adquery.log
Disable the IBM Security DirectControl Agent for Mac log:

% sudo /usr/local/share/centrifydc/bin/cdcdebug off

To enable logging with the IBM Security for Mac Diagnostic Tool:

Log in to the Mac as Local Admin and open the application MacDiagnosticTool.app.

The location of this app is “/Library/Application Support/Centrify/MacDiagnosticTool.app.” You can run the following command to open it:

% open /Library/Application Support/Centrify/MacDiagnosticTool.app
Click the Debug/Logs tab.
Click 0. Clear Debug Log Files.
Click 1. Enable Debugger.
Click 2. Get Start Time Point.

You do not need to remember the start time point; it will be saved automatically.
Click Quit to close the application.
Log out of the Local Admin account.
Reproduce the issue: try to log in as the affected Active Directory user. Let it fail.
Log back in as Local Admin and open the application MacDiagnosticTool.app again.
Click 4. Get End Time Point and enter input the affected Active Directory user name.
Click 5. Save Debug Log Files to Desktop, the tool will start to collect agent log files.

You might see a message display about installing the “otool” command; you can select Cancel or Install; either choice works.

The log file “CENTRIFY_FULL_LOG_PACK.zip” will be on the Desktop. Send the file to IBM Security Technical Support for analysis.
Click 6. Disable Debugger, then click Quit to close the application.

Enabling Logging for the Mac Directory Service

In addition to enabling logging for the agent, you may find it necessary to enable logging for the Open Directory Service.

To create a log file for the Open Directory Service:

Log in as or switch to the root or admin user.
Run the following command:

odutil set log debug

After running this command, you can find the resulting log files at: /var/log/opendirectoryd.log*. You can then provide both the agent log file and the Directory Service log file to IBM Security Support if you need assistance troubleshooting issues.

Using the Agent on a Dual-Boot System

If you are using a dual-boot system, and the computer name is the same for each version of the operating system, the IBM Security DirectControl Agent for Mac(adclient) will not launch when you reboot and switch operating systems. The problem is that each operating system sets its own password for adclient and the password does not work for the other operating system.

The best way to avoid this problem is to provide a different computer name for each operating system. Because the computer names are different, the password for one operating system is not changed by the other operating system.

If you want to use the same computer name for both operating systems, you can work around the problem, as follows:

Leave the domain (adleave) before rebooting and switching operating systems.

You may leave and join the domain after rebooting and switching the operating system. However, you will experience some delay while adclient attempts to launch and fails.
Reboot with the other operating system.
Rejoin the domain (adjoin).

Using adgpupdate Appropriately

If adgpupdate is run multiple times in succession, it is possible that not all group policies will be applied correctly. To avoid this problem, do not run adgpupdate more than once per minute.

Understanding Delays when Logging on the First Time with a New User Account

Depending on the configuration of your startup services, you may find that new users are unable to log on to a computer immediately (within the first 15 to 30 seconds) after a computer is rebooted.

By default, the Mac login window only requires the Disks and SecurityService startup services to start successfully to prompt for the user to log in. Authenticating users to Active Directory, however, requires the additional DirectoryServices startup service to be available. Starting the DirectoryServices startup service causes a 10 to 15 second delay before the LoginWindow can successfully authenticate new Active Directory users.

Configuring Single-sign on to Work with Non-Mac Computers

On a Mac computer, the ssh client does not forward (delegate) credentials to the server by default. Therefore, when attempting to use ssh from a Mac computer with DirectControl agent installed to a non-Mac computer with DirectControl agent installed, single sign-on (SSO) does not work. To fix this problem, set the configuration parameter, GSSAPIDelegateCredentials, to yes in the /etc/ssh_config file on the Mac computer.

Restricting Login Using FTP

In Active Directory, you can set properties to prevent a user from logging in to other Macintosh computers. However, this restriction will not prevent a user from logging in via FTP to Macintosh computers with the DirectControl agent installed. It does restrict logging in with telnet, ssh, rlogin, and rsh.

Logging on Using Localhost

For many UNIX platforms, you can log on using localhost to refer to the local computer; for example:

root@localhost

This syntax does not work when logging on to a Macintosh computer, whether using the Macintosh UI, or remotely through ssh or FTP.

Changing the Password for Active Directory Users

In the Mac OS X, the passwd command authenticates the user only after you type the user password. Because of this, the passwd command does not recognize the user as an Active Directory user until after the password is entered and the password prompts defined for Active Directory users, which are typically set through group policy or by modifying the IBM Security configuration file, are not displayed. You can still use the passwd or chpass command to change the Active Directory password for a user, but you will not see any visual indication that you are modifying an Active Directory account rather than a local user account.

Disabling the Apple Built-in Active Directory Plug-in

Apple provides a built-in Apple Directory plug-in that may interfere with the IBM Security DirectControl Agent for Mac installation and operation. Therefore, before installing the agent, disable Apple’s built-in Active Directory plug-in. In addition, remove Active Directory from the Authentication and Contacts search paths. If this plug-in is enabled and the IBM Security DirectControl Agent for Mac has been installed, disable the plug-in, then reboot the Macintosh computer for reliable operation.

To disable the Apple Directory plug-in and remove Apple Directory from the Authentication and Contacts search paths:

On a Mac computer, open the Directory Utility.

You can find the Directory Utility in one of these folders depending on the operating system that you are running:
- /System/Library/CoreServices
- /Applications/Utilities
Click the lock icon and enter credentials to allow you to make changes.
Click the Search Policy icon.
Click the Authentication tab, then select Custom path in the Search box.

If Active Directory was previously enabled, Active Directory appears in the Directory Domains box; for example:

/Active Directory/All Domains
Select /Active Directory/All Domains and click Remove — or select the minus - sign). Then click Apply.
Click the Contacts tab, then select Custom path in the Search box. If Active Directory was previously enabled, Active Directory shows (in red font) in the Directory Domains box; for example:

/Active Directory/All Domains
Select /Active Directory/All Domains and click Remove. Then click Apply.
Close the window.
If you have already installed the IBM Security DirectControl Agent for Mac, reboot the computer.

Showing the Correct Status of the IBM Security Plug-in

The IBM Security plug-in is automatically added to the list of Apple Directory Utility plug-ins that are used for lookup and authentication. However, if the Apple Directory Utility tool is running when you install the IBM Security DirectControl Agent for Mac, or when you join or leave a domain before updating to a new version of the agent, it will incorrectly display the status of the plug-in. For example, it will show the status as disabled, when in fact, the plug-in is enabled.

To avoid this problem, before launching the installer, be certain that the Apple Directory Utility tool is closed.

If the Directory Utility was open during installation, simply close and re-open Directory Utility, then make certain that the IBM Security plug-in is enabled.

You may also restart the IBM Security plug-in from the command line, as follows:

Close the Directory Utility.
Open a terminal.
Enter the following command:

/usr/local/share/centrifydc/bin/dsconfig restart
Open the Directory Utility. The status of IBM Security should be enabled.

Resolving VPN Access Issues with Mac OS X 10.7 and Later

Starting with Mac OS X 10.7, /etc/resolv.conf is no longer used for domain controller name resolution. Therefore, some VPN programs no longer update DNS server information in /etc/resolv.conf when signing on. On computers running Mac OS X 10.7 and later, this can result in the computer not being able to connect to a domain controller through a VPN.

To resolve this issue, explicitly specify in centrifydc.conf the location of DNS servers that are used to resolve domain controller names:

Open /etc/centrifydc/centrifydc.conf for editing.
Specify the IP addresses of DNS servers in the dns.servers parameter (if the parameter does not exist yet, create it now):
dns.servers: x.x.x.x y.y.y.y

where x.x.x.x y.y.y.y are the IP addresses of the DNS servers to use. This example shows two IP addresses; note that each IP address is separated by a space.
Save your changes to centrifydc.conf.
Restart the agent for the changes to take effect:

sudo /usr/local/share/centrifydc/bin/centrifydcrestart

Diagnosing Smart Card Login Problems

Two general methods for diagnosing smart card log in problems are provided:

By using the sctool utility as described in the sctool man page.
By performing the diagnostic procedures described in this section.

The following procedures are intended to diagnose multiple causes of smart card log in failure. It is recommended that you retest smart card login at regular intervals (such as after each step) as you perform this procedure.

Ensure that macOS built-in PIV token is not disabled.

% defaults read /Library/Preferences/com.apple.security.smartcard DisabledTokens

It should not exist.
Ensure that smart card support is enabled.

% sctool -s

It should show that smart card support is enabled.
Ensure that the smart card can be recognized by MacOS.

% sc_auth identities

It should show your card and the card has been paired to the Active Directory user.
Collect support information.

% sctool -S

Send the file /tmp/sctool.support to IBM Security Support.

Opening a Support Case Online

If you need assistance with troubleshooting an issue, you may need to open a case with IBM Security Support. Before opening a new case, IBM Security recommends searching the IBM Security Support Portal to see if your problem is a known issue or something for which there is a recommended solution.

To search the IBM Security Support Portal

Open https://www.delinea.com/support/ in a Web browser.
Click in the search field and type one or more key words to describe the issue, then click the search icon to view potential answers to your question.

If your issue is not covered in one of the search results, you should open a case with IBM Security Support.

To open a new support case

Log in to the IBM Security support portal.
Click Manage Cases, then click Open a New Support Case.

The NEW CASE DETAIL page appears.
Enter your case details, then click Next.

Provide as much information as possible about your case, including the operating environment where you encountered the issue, and the version of the IBM Security product you are working with.

A new page appears showing Suggest Knowledge Articles and Technical Resources. You can click Show More to see additional resources that might solve your problem.
Click No Thanks, Submit a Case to open a new case.

Collecting Information for Support Cases

To help ensure your issue gets resolved quickly and efficiently, gather as much information about your working environment as possible. See the information below in these two sections:

Collecting general information about your environment
Collecting information specific to login events

Collecting Information Specific to Smart Card Login Failure

Collect the following information prior to opening a support case related to smart card log in failure:

The smart card type (for example, PIV, CAC, CACNG, and so on), manufacturer, and model.
A screen image of the smart card and its certificates in Keychain Access.
The following log files:

/tmp/sctool_D.log

/tmp/adquery.log

/tmp/tokendfolder.log

/var/Centrify/tmp/adinfo_support.tar.gz

To generate these logs, run the following commands while logged in as the local administrator:

sctool -D > /tmp/sctool_D.log

adquery user -A username_of_smartcard_user > /tmp/adquery.log

sudo ls -l /System/Library/Security/tokend/ > /tmp/tokendfolder.log

sudo adinfo -t

Collecting General Information about Your Environment

Take the following steps to gather information about your working environment before opening a support case.

Verify that the DirectControl agent is running on the computer where you have encountered a problem. For example, run the following command:

ps aux | grep adclient

If the adclient process is not running, check whether the watchdog process, cdcwatch, is running:

ps aux | grep cdcwatch

The cdcwatch process is used to restart adclient if it stops unexpectedly.

The commands in the following three steps must be run as root or with the sudo command.
Enable logging for the DirectControl agent; for example:

sudo /usr/local/share/centrifydc/bin/cdcdebug on

Login events are captured in /var/log/centrifydc-login.log by default. Turning on cdcdebug captures login events in /var/log/centrifydc.log.
Create a log file for the Mac Directory Service. For example:
- To enable logging for opendirectoryd:
odutil set log debug
- To disable logging for opendirectoryd when sufficient log information is collected:
odutil set log default
Duplicate the steps that led to the problem you want to report. For example, if an Active Directory user can’t log in to a managed system, attempt to log the user in and confirm that the attempt fails. Be sure to make note of key information such as the user name or group name being used, so that IBM Security Support can identify problem accounts more quickly.
Verify that log file /var/log/centrifydc.log or /var/adm/syslog/centrifydc.log exists and contains data.
Run the cdcdebug command to generate logs that describe the domain and current environment; for example:

sudo /usr/local/share/centrifydc/bin/cdcdebug -f pack username

The following log files are created in /var/centrify/tmp when you execute the cdcdebug command:
- adinfo_support.tar.gz
- adinfo_support.txt
- cdcdebug.tar.gz
- dump_cache_error.log
- stacktrace.txt
If there is a core dump during or related to the problem, save the core file and inform IBM Security Support about it. IBM Security Support may ask for the file to be uploaded for review.

If the core dump is caused by a IBM Security process or command, such as adclient or adinfo, open the /etc/centrifydc/centrifydc.conf file and change the adclient.dumpcore parameter from never to always and restart the agent:

sudo /usr/local/share/centrifydc/bin/centrifydcrestart

For more information about starting and stopping the agent, see the Administrator’s Guide for Linux and UNIX.
If there is a cache-related issue, IBM Security Support may want the contents of the /var/centrifydc directory. You should be able to create an archive of the directory, if needed.
If there is a DNS, LDAP, or other network issue, IBM Security Support may require a network trace. You can use Ethereal to create the network trace from Windows or UNIX. You can also use Netmon on Windows computers.
Create an archive (for example, a .tar or .zip file) that contains all of the log files and diagnostic reports you have generated, and add the archive to your case or send it directly to IBM Security Support.
Consult with IBM Security Support to determine whether to turn off debug logging. If no more information is needed, run the following commands, which must be run as root or with sudo:

odutil set log default

sudo /usr/local/share/centrifydc/bin/cdcdebug off

Collecting Information Specific to Login Events

Login events are captured in /var/log/centrifydc-login.log by default. If you enable logging for the DirectControl agent by turning on cdcdebug, login events are then captured in /var/log/centrifydc.log.

The /var/log/centrifydc-login.log grows to a maximum size of 50M before it is compressed. When all compressed centrifydc-login.log files combined with the current log file exceed 250M, the oldest compressed log is replaced.