Hernán J. Larrea Symantec Corporation has not participated in the development of this document.  All sources will be properly mentioned at the end of the document.  I’m Hernán J. Larrea, an IT engineer with 8 years of experience in the field. I decided to address my career in Wintel technologies, specially Active Directory, Backup and Recovery environments, Storage solutions and a little programming in .NET. I’ve compiled this document since I believe it is very important for Netbackup administrators to count with this knowledge at the time of troubleshooting, Symantec has good articles and documents describing this processes, but compiling this in form of slides and animations helps to concentrate the eyes and the mind, in different and specific matters making learning much easier. If you want to know more about me, I invite you to visit my personal website http://www.hernanjlarrea.com.ar or my blog at http://blog.hernanjlarrea.com.ar and if you want to email me, you can do it at hjlarrea@hotmail.com. Hernán J. Larrea Things you need to know before thinking on the backup process. When the NetBackup master server boots up, a script automatically starts all services, daemons, and programs required by NetBackup.  The same is true on a media server: NetBackup automatically starts additional programs as required, including robotic daemons.  A daemon that executes on all servers and clients is the NetBackup client daemon, bpcd. On UNIX clients, inetd starts bpcd automatically so no special actions are required. On Windows clients, bpinetd performs the same functions as inetd.  Note that all NetBackup processes can be started manually by running the following:  /usr/openv/netbackup/bin/bp.start_ all Starting with NetBackup 6.0, the scheduler process bpsched is replaced by the following new services:  nbpem service (Policy Execution Manager): creates a persistent job worklist, starts each job when it is due (no wakeup interval), and sets a timer for the next due job.  nbjm service (Job Manager): accepts requests from nbpem to run backup jobs, or to run media jobs from commands such as bplabel and tpreq. nbjm acquires resources for each job, such as storage unit, drives, media, and client and policy resources, and executes the job.  nbrb service (Resource Broker): allocates resources in response to requests from nbjm. nbrb acquires physical resources from nbemm (the Enterprise Media Manager service), and manages logical resources such as multiplex groups, maximum jobs per client, and maximum jobs per policy. nbrb is also responsible for initiating drive unloads, and manages pending request queues. The NetBackup master server and the Enterprise Media Manager (EMM) server can be on the same physical host or on different hosts. Master server: responsible for running jobs as configured in NetBackup policies. The nbpem and nbjm services run only on the master server.  EMM server: allocates resources for one or more master servers. The EMM server is the repository for all device configuration information. The nbemm and nbrb services run only on the EMM server.  ▪ nbemm service: centralizes resource selection. nbemm maintains devices, media, and storage units in a relational database. Prior to NetBackup 6.0, resource selection was handled by bpsched, bptm, ltid, and the robotic device daemons. As you might already know, Netbackup operation implies communication across many, many daemons and processes, here you will find what happens when a backup starts.  Even the text speaks about tape and disk storage units, diagrams will only describe the process for tape (the process is only different only in one step, 15).  Even the text describes this is a backup on a UNIX host, process is just the same for Windows machines. Backup process After boot up: EMM & Master Server bprd 1) A start-up script launches bprd on the master server and ltid on the master server and all media servers. All other daemons and programs are started as necessary, including nbpem, nbjm, nbrb, and nbemm. 1 nbpem Media Server ltid Client 14 Backup process The policy execution manager service (nbpem) does the following: EMM & Master Server bprd 1 2 )Gets the policy list from bpdbm. 2 bpdbm nbpem 2 3) Builds a work list of all scheduled jobs. 6 nbjm 4 )Computes the due time for each job. 5 )Sorts the work list in order of due time. 6 ) Submits to nbjm all jobs that are currently due. Media Server Client 7 ) Sets a wakeup timer for the next due job. 8 ) When the job finishes, recomputes the due time of the next job and repeats at step 5. ltid 14 Backup process Next, the job manager service (nbjm) does the following: 9) Issues a single request (with a request ID) to nbrb, for all resources required by a job. nbrb gets the storage unit, tape drive, and media id information from nbemm and allocates client and policy resources. nbrb returns to nbjm an allocation sequence containing one allocation for each resource (each allocation contains a unique ID). nbrb also returns allocation data for the specific resource type. nbrb also returns the request ID along with the allocations so that nbjm can correlate the response with the right request (and job). EMM & Master Server bprd 1 bpdbm 2 nbpem 2 6 nbjm 9 nbemm 9 Media Server bpcd nbrb Client 10 10) nbjm starts the backup by using the client daemon bpcd to start the backup and restore manager bpbrm. bpbrm ltid 14 Backup process Next, the backup and restore manager (bpbrm) does the following: EMM & Master Server bprd 1 bpdbm 2 11) Starts bptm. nbpem 2 6 12) Starts the actual backup (or archive) by using the client daemon bpcd to start the backup and archive program bpbkar on the client. nbjm 9 nbemm 9 Media Server bpcd nbrb Client bpcd 10 bptm ltid 11 bpbrm bpbkar 14 Backup process Next, the backup and archive manager (bpbkar) does the following: EMM & Master Server 1 13) Sends information about files within the image to bpbrm, which directs the file information to the NetBackup file database. The information is sent by means of bpdbm on the master server. bpdbm 2 nbpem 2 6 nbjm 14) Transmits the backup image to bptm. This is accomplished in one of two ways : If the media server is backing up itself, bpbkar stores the image blockby-block in shared memory on the media server. If the media server is backing up a client on a different host, the bptm process on the server creates a child process of itself. The child receives the image from the client by means of socket communications and then stores the image block-by-block in shared memory on the server. bprd 9 nbemm 9 Media Server bpcd nbrb Client bpcd 10 bptm ltid 11 bpbrm 13 14 bpbkar Backup process Next, the backup manager for tape or disk does the following: 15) The bptm or bpdm process on the server takes the image from shared memory and directs it to the storage media. ■ If the storage media is tape, bptm requests information for the first media and drive to use, by exchanging information with nbjm. bptm sends mount requests for specific media and drives to the NetBackup Device Manager (ltid), which causes the media to be mounted on the appropriate devices. If, during the backup, a tape span is required, bptm again exchanges information with nbjm to release the correct tape and to get another one. nbjm exchanges information with nbrbto accomplish this. ■ For SharedDisk, AdvancedDisk, and OpenStorage: bptm requests the volume from nbjm, which passes the request to nbemm to choose the volume and media server to use. nbemmcalls nbrmms on the media server that was chosen to mount the volume. If a span is required, the same steps are used to mount the volume as described in the previous bullet. ■ For BasicDisk: bpdm writes the images to the path configured in the disk storage unit. The system disk manager controls the actual writing of data. In the case of an archive, bpbrm deletes the files from the client disk after the files have been successfully backed up. EMM & Master Server bprd 1 bpdbm 2 nbpem 2 6 nbjm 9 nbemm 9 Media Server bpcd nbrb Client bpcd 10 bptm ltid 11 bpbrm 13 14 bpbkar Backup process Next, the job manager service (nbjm) does the following: EMM & Master Server bprd 1 16) Receives completion status of the job from bpbrm. bpdbm 2 nbpem 2 6 17 17) Releases resources to nbrb and returns status to nbpem. nbjm 9 nbemm 9 Media Server bpcd nbrb Client bpcd 10 bptm ltid 11 bpbrm 13 14 bpbkar  Veritas NetBackup™ Troubleshooting Guide UNIX, Windows, Linux Release 6.5