Secrets of an AIX Administrator -Part 1

“How can I become an AIX systems administrator?”

As I reply back to all of these people and try to help those looking for assistance, I ponder the challenges newcomers to AIX administration must face. In the past, I could sense that some people were in dire straits and knew that things were going to turn out badly. For others, I could tell they had been given a bad task and knew that they would have to paddle upstream all the way. Some seemed just plain lost. But then I thought to myself, “Wouldn’t it be great if someone could share all of the experiences that can’t be gained from a classroom or training environment so that newcomers wouldn’t have to go through the struggles and mistakes that other administrators have made?”

With these experiences in mind, I knew I had to write this series of articles on the secrets of being an AIX systems administrator. My goal is to cover the things I wish I had known about AIX before I became an admin. These articles will cover the practical knowledge that can’t be acquired within training classes or books or gleaned from the man pages on a server. And, hopefully, it will save you from some of the headaches and heartaches I’ve faced in my many years of taking care of servers in all sorts of environments, from education and telecommunications to agriculture and outsourcing.

Breaking into the Business

To kick off this series, it’s best to start at the beginning. How does someone get into the position of being an AIX systems administrator? Do you have to have some accreditation from IBM? Is a computer science degree needed? Does a secret handshake or shadowy ritual grant you the powers of root access?

I’ve found that no two administrators took the exact same path to get where they are. On the one hand, I’ve met folks such as electrical engineers who have extensive programming knowledge, possess master’s degrees and become admins. On the other hand, there are plenty of people—like me—who come from non-technical focuses and have minimal formal training. (I majored in history and minored in political science, and took only one computer science class in college, which was primarily an introduction to Microsoft Windows and Office.) But there are some commonalities and steps that any admin will recommend for those interested in becoming an AIX systems administrator.

Check your motives. Ask yourself why you want to be an AIX systems administrator. There are plenty of rewards to becoming an admin, such as decent compensation, telecommuting, and flexible schedules; finding the right company or institution can make the job even better. But being an admin comes with plenty of pain, such as on-call rotations, long nights, off-hours work, and the perception that because you can telecommute, you can be contacted any time. So don’t charge lightly into this kind of commitment—you must first determine what’s important to you in your life.

Get some exposure. If you decide you want to become an AIX admin, start by getting some exposure to an AIX server. Ideally, you should find a simple system and have your own user ID for ­experimentation purposes. But servers aren’t typically available for just anyone to pick up; you’ll have to hunt one down. Your company might have a server available and some flexibility in setting up an account for you, but I’ve found that the easiest place to find access is through a university or a local community college. In most cases, the university will give you a bare-bones account with a few megabytes of storage for free just for taking some classes. You might have to contact the department that supports the servers to find out if they have an AIX server or other forms of UNIX, and you’ll need to be specific about what kind of access you’ll want.

Read all you can. There’s a tremendous wealth of free information out there on AIX systems administration. Publications such as POWER IT Pro keep people up-to-date with the latest news and information about AIX technology with practical tips and articles. IBM’s Redbooks on AIX are available on the web for free download in PDF format. And most admins will have a small library of books from publishers such as O’Reilly that they will lend out or recommend that you purchase. You can beef up your book knowledge quickly and easily, but real-world experience will always trump theory and handbooks.

Install Linux or play with other flavors of UNIX. Since its inception in the early 1990s, Linux has become hugely prolific. And with the advent of AIX 5L, there’s integration between AIX and Linux. Even if you have access to an AIX server, by building your own Linux box at home, you’ll be able to play around and experiment with commands as the root user—blowing a few things up in a safe environment—before you lay hands on an AIX production server that’s worth millions of dollars.

Find a mentor. Try to locate a skilled AIX admin who’s generous with sharing information and has a patient teacher’s spirit. Don’t ever think you know more than him or her; approach your admin with a kind, inquisitive attitude and learn as much as you can from seeing how he or she reacts to various situations over time. Take copious notes and ask questions whenever you can. And buy your mentor a meal or drink after work to show your appreciation for his or her knowledge and wisdom.

Apply and prepare for rejection. When trying to find a position, there’s a great paradox that happens to nearly everyone: You can’t get a job as an admin without experience, and you can’t get experience without a job. This is why you’ll need to beef up your resume by demonstrating all the time and energy you’ve invested into learning AIX. You might need to quantify this by passing a certification exam, getting references from qualified people, or finding a low-level entry position and working your way up from there. Be ready to get rejected, but don’t give up hope; stay determined and watchful.

If you follow these steps, you’ll be on the right path for a solid career in this field. And from there, you’ll understand more of the secrets of being an AIX systems administrator.

And what are the secrets, you ask? Let’s start with the all-important user ID.

User ID Utopia

One of the first things that AIX systems administrators get tasked with in order to familiarize themselves with their environments is user ID management. The theory is that this is one of the best ways for admins to get to know their user communities and the function and purpose of their servers. However, more often than not, user ID management is seen almost as a janitorial task—a sort of necessary evil that everyone has to take part in from time to time. That’s because it’s no fun working through endless password resets, changing huge groups of users who weren’t set up properly initially, or manually managing dozens of user IDs on many servers.

There are some good rules for managing user IDs that aren’t typically found in manuals or training classes. I’ve acquired these guidelines after years of seeing things go amiss, taking drastic measures to fix problems, and being kept awake at all hours of the night. In general, if you use these secrets, your life as an AIX admin will go much more smoothly.

Protect the root password. The more people who know the root password, the greater the likelihood of someone running the wrong command. Guard it as though it’s a bank account number.

Homogenize user IDs. Try to get a spreadsheet or system going to keep all of the user ID numbers for all of the user IDs across your servers in sync. This way, you won’t need to be brought in to set permissions or ownership should someone need to transfer files from one server to another or use an NFS mount across systems. If you’re administering more than just a few servers, consider using a utility such as NIS or LDAP to handle things from a single location.

Make important user IDs local. Make sure that any user IDs and home directories, such as those used by applications, are defined locally to the server. I’ve seen some environments in which network outages have crippled mission-critical servers because all of their resources were located externally to the systems. If your business won’t work without some specific user IDs being present, keep them local.

Establish all groups up front. One of the strangest problems an admin can encounter is getting the message, “Cannot set process credentials,” when someone attempts to log in. This message happens when the user’s primary group doesn’t exist on the server. To solve this problem and make management easier in the long run, give all the user IDs all of the possible groups they need up front.

Use proper home directory management. If you have user IDs that consume a good deal of space in their home directories, make the home directories into their own file systems, and then set permissions and ownership as you create the user IDs. Put the home directories in places in which they make sense; I’ve seen home directories placed in all sorts of locations such as \etc and \tmp, where they could potentially harm the server or cause general confusion. Keep them in \home or another file system structure that stands out for the applications on the server.

Set password expiration warnings. If you’re using best computer security practices or have to conform to federal laws or regulations, your user IDs will have regularly scheduled password expiration dates. Do yourself a favor and set a reminder email to go to your users a week or so before the change so that they can take action before they get locked out.

Install sudo. Lastly, I believe that there’s no greater application for managing user IDs than sudo. Sudo provides a safe and secure way to give access to privileged commands while minimizing risk to the server. The sudo configuration file can be ported across multiple servers and even onto different versions of AIX, UNIX, or Linux systems. I believe in using it on any server I administer, if for no other reason than providing another audit trail to becoming the root user.

Courtesy Christian Pruett &  POWER IT Pro.