IBMPlatformLSF家族安装和配置简介.V1.0

《IBMPlatformLSF家族安装和配置简介.V1.0》由会员分享，可在线阅读，更多相关《IBMPlatformLSF家族安装和配置简介.V1.0（52页珍藏版）》请在装配图网上搜索。

1、IBM Platform LSF家族安装和配置简介V1.0 版 马雪洁2013.5.7目录1 集群结构 11.1 单纯LSF环境（命令行提交） 11.2 LSF+PA（环境（WEB 提交）11.3 LSF+PM环境（PM 提交）32 LSF安装和基本配置举例32.1 LSF安装步骤32.1.1 获得LSF和 PAC安装包 32.1.2 解压缩 lsfinstall 安装脚本文件 42.1.3 修改 install.config 配置文件 42.1.4 执行安装 42.1.5 配置开机自启动 42.1.6 测试安装 42.1.7 启动/停止LSF进程（三种方式） 52.1.8 测试提交作业 62

2、.1.9 使能 root 提交作业 62.1.10 修改配置文件后 reconfig 62.1.11 日志和 debug 62.2 配置文件说明 62.3 常用命令 72.4 配置公平竞争调度策略 72.4.1 添加轮循调度队列 82.4.2 添加层次公平竞争策略 82.4.3 多队列公平竞争策略 82.4.4 使能配置 92.5 配置抢占调度策略 102.6 配置全局限制策略 102.6.1 限制用户运行的作业数目 102.6.2 限制节点运行作业数目 112.6.3 限制队列作业的运行限制 112.6.4 设定 General limits 112.6.5 使能配置 122.7 配置提交控

3、制脚本esub 122.8 配置资源管理 elim示例 132.8.1 汇报 home 目录空闲大小 132.8.2 汇报 root 进程数目 132.8.3 汇报应用程序许可证数目 132.8.4 测试 elim 脚本 142.8.5 添加资源定义和资源地图 142.8.6 查看资源数目 143 LSF命令行集成应用示例143.1 CFD+集成(spoolingfile) 153.1.1 CFD+安装和许可证 153.1.2 集成许可证管理 elim 153.1.3 添加 CFD+ job starter163.1.4 添加 CFD APP profile173.1.5 CFD+命令行提交脚

4、本实例 173.2 GAUSSIAN集成方式(spooling file) 183.2.1 Gaussian安装和许可证 183.2.2 Gaussia n命令行提交脚本实例 183.3 Abaqus的脚本集成(bsub 命令)183.4 Platform MPI 作业 193.5 Openmpi 作业 233.6 Intel MPI 作业 243.6.1 Express版本不记账方式243.6.2 Express版本 blaunch 记账方式263.6.3 Standard 版本 PAM 集成方式304 安装 PAC 335 使用PAC进行应用程序集成345.1 Gaussian 界面集成过

5、程 355.2 CFD+4集成后界面和后台脚本405.3 在PAC中监控许可证 426 安装 License Scheduler 446.1 基本安装测试 446.2 基本配置举例 446.2.1 添加许可证服务器地址 446.2.2 映射许可证 feature ： 446.2.3 使用许可证资源 456.2.4 配置许可证调度策略 457 常见问题 458 使用 man page 45售后技术支持 451集群结构较大的集群都会设计 单独的登录节点，用户只能 ssh到登录节点，不能直接 ssh到集 群的任何主节点和计算节点。同时配置用户在计算节点之间的ssh互信，为了并行作业的运行。登录节点也

6、安装LSF配置为LSF静态Client或者MXJ值为0,也即不运行作业的客户 端。集群的 WEB节点与办公访问局域网一个网段。如需使用浮动client，主节点网卡需要1.1单纯LSF环境（命令行提交）#|Page#|Page访问网络DesktopLSF Float Clie ntDesktopLSF Float Clie ntDesktop LSF Float Clie nt作业提交脚本 设计流程脚本 bsubjobsDesktopLSF Float Clie nt#|PageSSHLSF主节点（可扩展到3个）SSH作业提交脚本 设计流程脚本bsubjobs登录节点F Static Clie

7、nt用户隔离计算资源，W3脚本流程中的” bsub jobs 将作业散到*集群计算节点。#|Page#|Page管理网络#|Page#|Page1.2 LSF+PAC 环境（WEB 提交）用户通过portal提交作业：3|Page1.3LSF+PM环境（PM提交）LSF主节点亀Process Man ager Serve登录节点(WEB PORTAL)Linux于洗予熬高性能集群W严、管理网络存储网络4|Page#|Page2 LSF安装和基本配置举例2.1安装前的准备工作NIS ready; NFS/GPFS ready2.2 LSF安装步骤Use root to in stall.Get

8、NIS and NFS/GPFS ready.2.2.1 获得LSF和PAC安装包Isf8.3_li nux2.6-glibc2.3-x86_64.tar.ZIsf8.3 lsfinstall linux x86 64.tar.Z pac8.3_sta ndard_li nu x-x64.tar.Z 许可证文件 platform_hpc_std_entitlement.dat222解压缩Isfinstall安装脚本文件Put the package un der /root/lsfrootS2 lsf# gun zip Isf8.3_lsfi nstall_li nux_x86_64.tar.

9、Ztar -xvf lsf8.3_lsfi nstall_li nux_x86_64.tar2.2.3 修改 install.config配置文件首先添加集群管理员lsfadmin。cd lsf8.3_lsfi nstallvi in stall.c onfigrootS2 lsf8.3_lsfinstall# cat install.configLSF_TOP=/opt/lsf（安装目录）LSF_ADMINS=lsfadmin先创建 lsfadmin 的用户名）LSF_CLUSTER_NAME=platformf集群名称，任意指定）LSF_MASTER_LIST=s2 s3 （LSF理节点）

10、LSF_ENTITLEMENT_FILE=/root/lsf/platform_hpc_std_entitlement.dat（安装源许可证的地址）LSF_TARDIR=/root/lsf/（安装源文件包的地址）2.2.4 执行安装./lsfi nstall -f in stall.c onfig2.2.5 配置开机自启动/opt/lsf/9.1/i nstallhostsetuprhostsetup2.2.6 测试安装安装目录下的/conf目录rootS2 conf# source profilesfAdd source profile .lsf to /e tc/profile if no

11、 rsh, set ssh in Isf.c onfrootS2 conf# tail Isf.co nfLSF_RSH=ssh2.2.7 启动/停止LSF进程（三种方式）rootS2 conf# lsfstartup/lsfstop或者lsadm in limstatup/limshutdow nlsadm in resstartup/resshutdow nbadm in hstartup/hshutdow n或者lsf_daem ons start/stoprootS2 conf# lsidIBM Platform LSF Express 8.3 for IBM Platform HPC

12、, May 10 2012Copyright Platform Computing Inc., an IBM Company, 1992-2012.US Gover nment Users Restricted Rights - Use, duplicati on or disclosure restricted by GSA ADP Schedule Con tract with IBM Corp.My cluster n ame is platformMy master n ame is s2You have new mail in /var/spool/mail/rootrootS2 c

13、onf# lsloadHOST_NAMEstatus r15sr1mr15mutpg lsit tmpswp mems2ok0.00.00.01%0.010 151G20G61Gs4ok0.00.00.02%0.012 183G20G62Gs6ok0.00.00.03%0.012 3734M2G30Gs5ok0.00.00.05%0.012 3468M2G30G228测试提交作业bsub sleep 100000 2.2.9 使能root提交作业en able root to submit job:LSF_ROOT_REX=local重启LSF进程。2.2.10修改配置文件后reconfig修

14、改lsf.*配置文件后lsadmin reconfig修改lsb.*配置文件后badmin reconfig部分参数需要重启LSF主调度或者其他进程：badmin mbdrestart; lsadmin limrestart;lsadm in resrestart; badm in hrestart2.2.11 日志和 debugFind the logs un der log directory. LSF will run mainly 3 processes on each no de, on master node will have 2 more.Master: lim,res,sba

15、tchd,mbatchd,mbschedCompute:lim,res,sbatchdTurn on debug in comma nd line:Run lim -2 directly on node to check why lim not startup.2.3配置文件说明目录 /e tc/i nit.d:/etc/i nit.s/lsflsf服务自启动脚本目录 /apps/platform/8.3/lsf/c onf:lsf.c onfIs配置文件lsf.cluster.cluster83 集群配置文件lsf.shared共享资源定义文件./lsbatch/cluster83/co n

16、figdir/lsb.* 调度系统配置文件Isb.queuesIsf队列配置文件Isb.paramsIsf调度参数配置文件Isf用户与用户组配置文件Isb.applications Isf 应用配置文件Isb.hostsIsf机器与机器组配置文件Isb.resourcesIsf资源配置文件Isb.modulesIsf模块配置文件Isb.users2.4常用命令bsub:提交作业；bjobs:查看作业信息；bhist:查看作业历史；Ishosts:查看节点静态资源；bhosts， IsIoad:查看节点状态和资源信息;bqueues :查看队列配置；bIimits :查看限制Iimit信息；Is

17、id:集群版本和主节点；bmod:修改 bsub option ；2.5基于资源的调度策略bsub -R “ (type=LINUX2.4 & r1m 2.0)|(type=AIX & r1m 1.0) ” 或者在队歹V Isb.queues或者Isb.application 文件定义：RES_REQ=select(type=LINUX2.4 & r1m 2.0)|(type=AIX & r1m =300 & mem500 orderswap:memrusageswap=300,mem=500 job1bsub -R rusagemem=500:app_lic_v2=1 | mem=400:a

18、pp_lic_v1.5=1 joblbsub -R selecttype=a ny & swp=300 & mem500 ordermem jobl2.6配置公平竞争调度策略2.6.1 添加轮循调度队列Modify lsb.queues, add follow ingBegi n QueueQUEUE_NAME = rou ndRobinPRIORITY = 40FAIRSHARE = USER_SHARESdefault,1#USERS = userGroupA Define your own usergroupEnd QueueRun badm in reconfig to en able

19、 the cha nge.Run bqueues - to check the queue sconfigure2.6.2 添加层次公平竞争策略Add following queue to add hierarchicalshare policy:Begi n QueueQUEUE_NAME = hierarchicalSharePRIORITY = 40USERS = userGroupB userGroupCFAIRSHARE = USER_SHARESuserGroupB,7 userGroupC,3End Queue2.6.3 多队列公平竞争策略在lsb.queues中添加下列队列，注

20、意节点组和用户组定义。Begi n QueueQUEUE_NAME = verilogDESCRIPTION = master queue defi niti on cross-queuePRIORITY = 50FAIRSHARE = USER_SHARESuser1,100 defaultFAIRSHARE_QUEUES = normal shortHOSTS = hostGroupC # resource contention #RES_REQ = rusageverilog = 1End QueueBegi n QueueQUEUE_NAME = shortDESCRIPTION =

21、short jobs PRIORITY = 70highestHOSTS = hostGroupCRUNLIMIT = 5 10End QueueBegi n QueueQUEUE_NAME = normalDESCRIPTION = default queuePRIORITY = 40# lowestHOSTS = hostGroupCEnd Queue2.6.4 使能配置badm in reconfig提交作业，并查看队列的用户动态优先级变化:bqueues -l normal11|Page2.7配置抢占调度策略配置最基本的slots抢占：Begi n QueueQUEUE_NAME =

22、shortPRIORITY = 70HOSTS = hostGroupC# pote ntial co nflictPREEMPTION = PREEMPTIVE no rmalEnd QueueBegi n QueueQUEUE_NAME = normalPRIORITY = 40HOSTS = hostGroupC# pote ntial c on flictPREEMPTION = PREEMPTABLEshortEnd Queue向两个队列提交作业，查看被preempt的作业的pending原因。2.8配置全局限制策略2.8.1限制用户运行的作业数目在lsb.users文件中添加：Be

23、gi n User12 | Page# | PageUSER_NAMEMAX_JOBS JL/P# | Page# | Pageuser1 4user221user3-2 groupA 8 groupBDefaultEnd User282限制节点运行作业数目在Isb.hosts文件中：Begi n HostHOST_NAME MXJ JL/Uhostl42host221host3!-End Host2.8.3 限制队列作业的运行限制在lsb.queues中添加：Begi n QueueQUEUE_NAME = myQueueHJOB_LIMIT = 2PJOB_LIMIT = 1UJOB_LI

24、MIT = 4HOSTS = hostGroupAUSERS = userGroupAEnd Queue2.8.4 设定 Ge neral limits在lsb.resources 文件定义全局 general limits 示例：Begi n LimitUSERS QUEUES HOSTS SLOTS MEM SWP13 | Page# | Pageuser1hostB -20%14 | Pageuser2 no rmal hostA2015 | PageEnd LimitBegi n LimitNAME = limitlUSERS = userlPER_HOST = hostA hostC

25、TMP = 30%SWP = 50%MEM = 10%End LimitBegi n LimitPER_USER QUEUES HOSTS SLOTS MEM SWP TMPSgroupA - hgroup1 -2user2 normal -200- short200End Limit2.8.5 使能配置badm in reconfig2.9配置提交控制脚本esub全局esub脚本在作业被提交是调用，可以被自动的或者显式的调用从而控制用户作业提 交的行为。编辑esub.project文件在$LSF_SERVERD下面（chmod为可执行）：#!/bi n/shif _$LSB_SUB_PARM

26、_FILE != _ ; the n.$LSB_SUB_PARM_FILEif _$LsB_SUB_PROJECT_NAME = _ ; the necho You must specify a project! &2exit $LSB_SUB_ABORT_VALUEfi fiexit 0在 Isf.conf 中定义 LSB_ESUB_METHOD”roject ”2.10配置资源管理elim示例2.10.1汇报home目录空闲大小编辑elim文件elim.home，放置在$LSF_SERVERD下面。chmod为可执行。#!/bi n/sh while true ; dohome=df -k

27、 /home | tail -1 | awk pri ntf %4.1f, $4/(1024*1024)、echo 1 home $homesleep 30done2.10.2汇报root进程数目编辑elim.root，放置在$LSF_SERVERD下面。chmod为可执行。#!/bi n/sh while true ; doroot=ps -ef | grep -v grep | grep -c Aroot echo 1 rootprocs $rootsleep 30done2.10.3汇报应用程序许可证数目#!/bi n/sh lic_X=0 ; num=0while true ; do#

28、 only want the master to gather lic_Xif $LSF_MASTER = Y ; the nlic_X=lmstat -a -c lic_X.dat | grep .&2fi# only want tra inin g8, training1 to gather simpt on lice nsesif host name = tra inin g8 -o host name = tra inin g1 ; the nnum=lmstat simpt on _lic.dat | grep . &2fi# all hosts in cludi ng master

29、, will gather the follow ingroot=ps -efw | grep grep | grep -c root1 &2tmp=df /var/tmp | grep var | awk print $4 /1024、 &2if $LSF_MASTER = Y ; the necho 4 lic_X $lic_X simpt on $num rtprc $root tmp $tmpelseecho 3 simpt on $num rtprc $root tmp $tmpfi# the same INTERVAL values defi ned in lsf.sharedsl

30、eep 60done2.10.4测试elim脚本直接运行./elim.root查看elim输出是否正确。2.10.5添加资源定义和资源地图在lsf.shared文件中添加rootprocs定义，并在lsf.cluster resources Map中添加资源和节点 的映射关系。使能配置：lsadm in recon fig; badm in reconfig2.10.6查看资源数目lsload -3 LSF命令行集成应用示例本节例举几个应用的不同集成方式。使用spooling文件或者bsub命令行都可以自由转换3.1 CFD+集成(spooling file )3.1.1 CFD+安装和许可证

31、安装路径：ln-3620-4许可证：/gpfs/software/cfdpp/mbi n/Metacomp.lic许可证服务器：ln-3620-4启动许可证服务器：hpcadmi nmn-3650 jessi$ ssh ln-3620-4Last login: Tue Mar 26 19:19:24 2013 from mn-3650.private.dns.zonehpcadm inln-3620-4 $ /gpfs/software/cfdpp/mbi n/lmgrd -c/gpfs/software/cfdpp/mbi n/Metacomp.lic确认许可证服务器是否正常运行：/gpfs

32、/software/cfdpp/mb in/lmutil lmstat -a -c /gpfs/software/cfdpp/mbi n/Metacomp.lic3.1.2 集成许可证管理elim添加elim方法：(elim全集群只需运行一个，因此只在头节点放置elim脚本即可)在头节点：cd $LSF_SERVERDIR添加如下文件：elim.lic:rootm n-3650 jessi# cd $LSF_SERVERDIRroot mn-3650 etc# pwd/opt/lsf/8.3/li nux2.6-glibc2.3-x86_64/etcroot mn-3650 etc# cat

33、elim.lic#!/bi n/shtotallice nces=/gpfs/software/cfdpp/mbi n/lmutil lmstat -a -c/gpfs/software/cfdpp/mbi n/Metacomp.lic |grep Users of CFD+_SOLV_Ser | /bi n/cut -d -f7 while true dousedlice nces=/gpfs/software/cfdpp/mbi n/lmutil lmstat -a -c/gpfs/software/cfdpp/mbi n/Metacomp.lic | /bi n/grep Users o

34、f CFD+_SOLV_Ser | /bi n/cut -d -f13cfd_lic=$( $totallice nces-$usedlice nces)echo 1 cfdic $cfdic/bin/sleep 30doneroot mn-3650 etc# chmod a+x elim.lic修改如下的配置文件：rootmn-3650 etc# vi $LSF_ENVDIR/lsf.shared添加如下一行：cfd_lic Numeric 30 Y(CFD+ Lice nse)rootm n-3650 etc# vi $LSF_ENVDIR/lsf.cluster 在resourcemap

35、 一段添加如下一行：Begi n ResourceMapRESOURCENAME LOCATIONcfd_licallhostiddefa ult root mn-3650 etc# lsadm in rec on fig; badm in reconfig3.1.3 添加 CFD+ job starter如果使用spooling file可不用添加。(Portal集成方式使用)添加jobstarter可执行文件：hpcadm inmn-3650 jessi$ cat /opt/lsf/jobstarter/cfd_starter#!/bi n/shMPI_RUN=/gpfs/software

36、/cfdpp/hpmpi/bi n/mpir uncase $PRESSION inSINGLE_PRESSION)CFD_CMD=/gpfs/software/cfdpp/mbi n/mcfd.11.1/r4_hpmpimcfdJ JDOUBLE_PRESSION)CFD_CMD=/gpfs/software/cfdpp/mbi n/mcfd.11.1/hpmpimcfdJ JesacCMD=$* -hostfile $LSB_DJOB_HOSTFILE $CFD_CMDeval $CMD3.1.4 添加 CFD APP profilerootmn-3650 etc# vi $LSF_ENV

37、DIR/lsf.shared添加如下配置：Begi n Applicati onNAME = cfdJOB_STARTER=/opt/lsf/jobstarter/cfd_starterRES_REQ=rusagecfd_lic=1End Applicati onbadmnin reconfig使得此文件生效，使用bapp - cfd查看是否成功:root mn-3650 bin# bapp -l cfdAPPLICATION NAME: cfd-No descripti on provided.STATISTICS:NJOBS PEND RUN SSUSP USUSP RSV12 12 0

38、0 0 0PARAMETERS:JOB_STARTER: /opt/lsf/jobstarter/cfd_starterRES_REQ: rusagecfd_lic=13.1.5 CFD+命令行提交脚本实例hpcadm inmn-3650 jessi$ cat cfd.sh#!/bi n/sh#BSUB -n 12#BSUB -o %J.out#BSUB -e %J.err#BSUB -app cfd#BSUB -R rusagecfdl_l ic=1cd /gpfs/software/cfd+/test/ogive/gpfs/software/cfdpp/hpmpi/bi n/mpiru n

39、 -hostfile $LSB_DJOB_HOSTFILE /gpfs/software/cfdpp/mbi n/mcfd.11.1/hpmpimcfd然后bsub cfd.sh提交作业。3.2 GAUSSIAN 集成方式(spooling file )3.2.1 Gaussian安装和许可证路径：/gpfs/software/Gaussia n/许可证：无许可证版本，单个作业只能单机运行。3.2.2 Gaussian命令行提交脚本实例下面脚本：g03.sh#!/bi n/sh#BSUB -q qchem#BSUB -n 4#BSUB -R spa n hosts=1#BSUB -cwd .#

40、BSUB -e %J.err#BSUB -o %J.outJOB=Full_codes_112_ipr_C1_ JOBNAME=base name $JOB .comexport g03root=/gpfs/software/Gaussia nexport GAUSS_SCRDIR=/tmpsource $g03root/g03/bsd/g03.profile/gpfs/software/Gaussia n/g03/g03 $JOBNAME.log提交作业:bsub g03.sh3.3 Abaqus的脚本集成(bsub命令)编辑脚本abaqus_run.sh脚本#!/bi n/sh# vers

41、io n: 1.3.0export ABAQUS_CMD=/gpfs/software/Abaqus/Comma nds/abaqusexport LM_LICENSE_FILE=/gpfs/software/Abaqus/Lice nse/abq612.lic#指定cpu number,注意要与bsub命令行中-n指定的cpu个数一致export NCPU=16#指定输入文件export INPUT_FILE=beam.i np#指定作业名export JOB_NAME=abaqusob3$ABAQUS_CMD job=$JOB_NAME cpus=$NCPU in put=$INP_INP

42、UT_FILE2）通过LSF提交#进入输入数据所在目录，执行bsub命令bsub -q qeng -n 16 ./abaqus_r un.sh3.4 Amber作业（blaunch集成，可记账）针对intelmpi，编写mpdboot.lsf脚本。变为可执行，放置在 $LSF_SERVERD下面编写提交作业脚本：ymeim nis test$ cat n ew.sh#!/bi n/sh#BSUB -q small#BSUB -n 128#BSUB -o %J.out#BSUB -e %J.err#BSUB -J IMPI#BSUB -x#export PATH=/gpfs01/software

43、/i ntel/impi/4.1.0.024/i ntel64/bi n:$PATH#/gpfs01/home/ymei/jessi/mpdboot.lsfmpdboot.lsfexport l_MPI_DEVICE=ssm#export l_MPI_FABRICS=shm:ofa#export l_MPI_FAST_STARTUP=1#export I_MPI_DEVICE=rdssm#mpiexec -np $LSB_DJOB_NUMPROC /gpfs01/software/in tel/impi/4.1.0.024/test/hellowordmpiexec -np $LSB_DJOB

44、_NUMPROC $AMBERHOME/bi n/san der.MPI -ng 32 -groupfile remd10.groupfilempdallexit提交作业：bsub n ew.sh3.5 Platform MPI 作业3.5.1 安装 Platform MPI确认用户无密码访问 ssh OK)安装Platform MPI到共享目录下:sh platform_mpi-08.3.0.0-0320r.x64.sh -installdir=/opt/pmpi-norpm如果缺失C Compiler，执行: yum in stall gcc3.5.2 LSF外面验证安装 OK设置环境变量

45、：export MPI_REMSH=ssh -x export MPI_ROOT=/opt/pmpi/opt/ibm/platform_mpi/ 编译helloworld示例程序：/opt/pmpi/opt/ibm/platform_mpi/bi n/mpicc -o helloworld/opt/pmpi/opt/ibm/platform_mpi/help/hello_world.crootserver3 help# /opt/pmpi/opt/ibm/platform_mpi/bi n/mpiru n -f ./help/hostswarning: MPI_ROOT /opt/pmpi/o

46、pt/ibm/platform_mpi/ != mpiru n path /opt/pmpi/opt/ibm/platform_mpiHello world! Im 1 of 4 on server3Hello world! Im 0 of 4 on server3Hello world! Im 3 of 4 on computer007Hello world! Im 2 of 4 on computer007rootserver3 help# cat ./help/hosts-h server3 -np 2 /opt/pmpi/opt/ibm/platform_mpi/help/hellow

47、orld-h computer007 -np 2 /opt/pmpi/opt/ibm/platform_mpi/help/helloworld3.5.3 通过LSF提交export MPI_REMSH=bla unch$ mpirun -np 4 -IBV /helloworld$ mpirun -np 32 -IBV /helloworld$ mpirun -np 4 -TCP /helloworld或者rootserver3 conf# bsub -o %J.out -e.%J.err -n 4/opt/pmpi/opt/ibm/platform_mpi/b in/mpiru n -lsb

48、_mcpu_hosts /opt/pmpi/opt/ibm/platform_mpi/help/helloworldJob is submitted to default queue vno rmal.rootserver3 conf# bjobsJOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME210 root PEND normal server3*elloworld May 9 10:55rootserver3 conf# cat 210.outSen der: LSF System vjessicomputer0

49、07Subject: Job 210: in cluster DoneJob was submitted from host by user in cluster .Job was executed on host(s) , in queue , as user in cluster . was used as the home directory. was used as the worki ng directory.Started at Thu May 9 18:49:06 2013Results reported at Thu May 9 18:49:07 2013Your job lo

50、oked like:# LSBATCH: User in put/opt/pmpi/opt/ibm/platform_mpi/b in/mpir un -lsb_mcpu_hosts/opt/pmpi/opt/ibm/platform_mpi/help/helloworldSuccessfully completed.Resource usage summary:CPU time :0.23 sec.Max Memory :2 MBAverage Memory :2.00 MBTotal Requested Memory :-Delta Memory :-(Delta: the differe

51、 nee betwee n total requested memory and actual max usage.)Max Swap :36 MBMax Processes :1Max Threads :1The output (if any) follows:Hello world! Im 2 of 4 on computer007Hello world! Im 0 of 4 on computer007Hello world! Im 1 of 4 on computer007Hello world! Im 3 of 4 on computer007PS:Read file v.210.e

52、rr for stderr output of this job.或者更多参数$ /opt/platform_mpi/bi n/mpiru n -np 120 -ibv -hostlist cn-22-001 cn-22-002 cn-22-003 cn-22-004 cn-22-005 cn-22-006 cn-22-007 cn-22-008 cn-22-009 cn-22-010 /data/hello_world如果希望MPI作业不通过LSF提交运行，修改MPI_USELF环境变量为n3.6 Openmpi 作业下载 openmpi软件包./con figure LIBS=-ldl -

53、with-lsf=yes -prefix=/usr/local/ompi/Ope n mpi1.3.2之上版本已经于 LSF blau nch紧密集成。提交 openmpi作业：bsub -n2 -o %J.out -e %J.err mpiexec mympi.out3.7 In tel MPI 作业3.7.1 Express版本不记账方式如果需要对作业记账，需要使用blaunch的集成方式。环境变量设置.bsahrcexportPATH=/gpfs/software/i ntel/composerxe/bi n/:/gpfs/software/i ntel/mpi_41_0_024/i n

54、clude:/gpfs/software/i ntel/mpi_41_0_024/bi n64:/gpfs/software/i ntel/composerxe/mkl:$PATHsource /gpfs/software/i ntel/composerxe/bi n/compilervars.sh in tel64source /gpfs/software/i ntel/mpi_41_0_024/bi n64/mpivars.shsource /gpfs/software/i ntel/composerxe/mkl/b in/mklvars.sh in tel64MPI测试程序Hellowo

55、rld.c#i nclude mpi.h#in clude #in clude int main (i nt argc,char*argv)int myid, nu mprocs;int n amele n;char processor_ nameMPI_MAX_PROCESSOR_NAME;MPI_I nit(&argc, &argv);MPI_Comm_ra nk(MPI_COMM_WORLD,&m yid);MPI_Comm_size(MPI_COMM_WORLD,&nu mprocs);MPI_Get_processor_ name(processor_ name,&n amele n

56、);fprintf(stderr,Hello World! Process %d of %d on %sn,myid, numprocs, processor_name);MPI_Fi nalize();命令执行，TCP协议mpir un -machi ne host.eth -env l_MPI_FABRICS shm:tcp -np 32 ./helloworld.icc命令执行，IB网络mpiru n -machi ne host.ib -env I_MPI_FABRICS shm:ofa -np 32 ./helloworld.icc命令执行，Debug模式mpir un -machi

57、 ne host.ib -env I_MPI_FABRICS shm:ofa -env l_MPI_DEBUG 2 -np32 ./helloworld.iccLSF提交脚本bsubn telmpi_ib.sh#!/bi n/sh#BSUB -cwd .#BSUB -R spa n ptile=4#BSUB -e %J.err#BSUB -o %J.outmpiru n-machi ne$LSB_DJOB_HOSTFILE -envI_MPI_FABRICSshm:ofa ./helloworld.icc提交作业：bsub bsub_ in telmpi_ib.sh3.7.2 Express 版本 blaunch 记账方式3.721 编写 mpdboot.lsf 文件#!/usr/b in/env pyth oniiiiiimpdboot for LSF-f | -hostfile hostfile-i | -ifh n=alter nate_ in terface_host name_of_ip_address-f | -hostfile hostfile-hiiiiiiimport reimport stri ngimport timeimport sysimport getoptfr