HPC

From ISOR
(Difference between revisions)
Jump to: navigation, search
(Created page with "The High Performance Cluster of WiWi == How To for users == == How To for Group Admins ==")
 
(implementation)
 
(17 intermediate revisions by one user not shown)
Line 1: Line 1:
 
The High Performance Cluster of WiWi
 
The High Performance Cluster of WiWi
 
== How To for users ==
 
== How To for users ==
 +
* Approach your group admin (see list below) to create an account for you, or change your password if you have forgotten your old one.
 +
* Your group admin will come back to you with your account (username and password). A user '''Mustermann''''s account (member of '''Arbeitsgruppe TestAG''', '''ag_te''') might be created as '''ag_te-mustermann'''.
 +
* From within the university's network (or [https://zid.univie.ac.at/vpn/ VPN] if working from home), the user '''Mustermann''' might use the command line to connect to the HPC:
 +
ssh  ag_te-mustermann@hpc3.wiwi.univie.ac.at
 +
If a session is connected for the first time, just accept the fingerprint question and continue with your password. Your home directory will be created. Once logged in, your remote prompt will look like:
 +
[ag_te-mustermannl@hpc3 ~]$
 +
Now you can run your commands; e.g.
 +
[ag_te-mustermannl@hpc3 ~]$ cd; pwd
 +
that will return the name of your home directory.
 +
 +
If you would like to transfer files into your home-directory, you could use a [https://zid.univie.ac.at/en/online-storage-space/user-guides/sftp/ sftp client] (like WinSCP, Cyberduck, ...).
 +
 +
You should clear up once your computing is done. In case you want some files to be shared with your group members for later use, you could use /group.storage/ag_testAG; whereas ag_testAG has to be replaced with your ag_Arbeitsgruppe.
 +
[ag_te-mustermannl@hpc3 ~]$  cd /group.storage/ag_testAG
 +
ag_Arbeitsgruppe might be created on the fly so that you can change into it. Physically, you are at another, slower machine, which could be shown by
 +
[ag_te-mustermann@hpc3 ag_testAG]$ df -h .
 +
  Dateisystem                  Größe Benutzt Verf. Verw% Eingehängt auf
 +
  172.16.1.101:/home/ag_testAG  91T    1,6T  90T    2% /group.storage/ag_testAG
 +
 +
* How to run your jobs, see [[Slurm]].
 +
 +
 +
<!--If you would like to change your password, please see the section '''step by step''' .-->
  
 
== How To for Group Admins ==
 
== How To for Group Admins ==
 +
Groups are independently administered by their group admins.
 +
To indicate the group-spaces, the User-IDs (uid) are the Group IDs (gid) plus some addition.
 +
 +
I would suggest prefixing the users with ag_(+2 letters of group).
 +
 +
A new user in ag_hautsch should be ag_ha-USERXYZ. Her uid=7000+X, and her home-directory /home/ag_ha-USERXYZ.
 +
 +
 +
===Specs===
 +
Gruppe Prof. Krivobokova: Hr. Stefan Geissler (stefan.geissler@univie.ac.at)
 +
Gruppe Prof. Cuchiero: Hr. Guido Gazzani (guido.gazzani@univie.ac.at)
 +
Gruppe Prof. Ehmke: Hr. Rico Kötschau (rico.koetschau@univie.ac.at)
 +
Gruppe Prof. Dörner: Roland Braune, Georg Fröhlich (georg.erwin.adrian.froehlich@univie.ac.at)
 +
Gruppe Operations Research: Michael Kahr (m.kahr@univie.ac.at)
 +
===implementation===
 +
ag_testAG_ou        ag_testAG      (gid=9000) andreas.loibl@univie.ac.at                  uid=9000 + X
 +
 +
                                          DN: uid=ag_ha-loibl,ou=ag_hautsch_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at
 +
                                            DN: all the other admins as well
 +
 +
ag_krivobokova_ou  ag_krivobokova (gid=8000) stefan.geissler@univie.ac.at                uid=8000 + X
 +
 +
                                          DN: uid=ag_kr-geissler,ou=ag_krivobokova_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at
 +
 +
ag_cuchiero_ou      ag_cuchiero    (gid=3000) guido.gazzani@univie.ac.at                  uid=3000 + X 
 +
 +
                                          DN: uid=ag_cu-gazzani,ou=ag_cuchiero_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at
 +
 +
ag_ehmke_ou        ag_ehmke      (gid=4000) rico.koetschau@univie.ac.at                uid=4000 + X
 +
 +
                                            DN: uid=ag_em-koetschau,ou=ag_ehmke_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at
 +
 +
ag_doerner_ou      ag_doerner    (gid=5000) georg.erwin.adrian.froehlich@univie.ac.at  uid=5000 + X
 +
 +
                                            DN: uid=ag_do-froehlich,ou=ag_doerner_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at
 +
 +
                                            roland.braune@univie.ac.at
 +
                                            DN: uid=ag_do-braune,ou=ag_doerner_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at
 +
 +
ag_operres_ou      ag_operres    (gid=6000) m.kahr@univie.ac.at                        uid=6000 + X
 +
 +
                                            DN: uid=ag_op-kahr,ou=ag_operres_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at
 +
 +
ag_hautsch_ou      ag_hautsch    (gid=7000) andreas.loibl@univie.ac.at                  uid=7000 + X
 +
 +
                                            DN: uid=ag_ha-loibl,ou=ag_hautsch_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at
 +
In case any admin wants to use SSH (instead of the ldap web interface), only the '''uid''' would be necessary to log in (within the uni's network):
 +
ssh ag_ha-loibl@hpc3.wiwi.univie.ac.at
 +
 +
=== Step by Step ===
 +
Group admin is within the university's network (or using VPN if working from home).
 +
 +
If you want to test a bit, '''ag_testAG_ou''' is the playground to do so.
 +
* Group Admin can reach the admin interface: https://hpc3.wiwi.univie.ac.at
 +
* Group Admin can log into the Web-Interface , e.g.
 +
uid=ag_ha-loibl,ou=ag_hautsch_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at
 +
password 12345
 +
[[File:Login.png|none|300px|Login]]
 +
If you might want to change your password, click your uid:
 +
[[File:Change-password.png|none|300px|Change Password]]
 +
* Connect via ssh:
 +
ssh ag_ha-loibl@hpc3.wiwi.univie.ac.at
 +
* Within this ssh session, Group admin can access OpenLdap directory entry (not the local /etc/passwd!)
 +
getent passwd | grep ag-XY
 +
[[File:Ssh-session.png|none|300px|SSH]]
 +
If you assigned an uid of 9001, 9002, or 9003 to a new user, or /home/ag_te-testuser-1 as a home directory, or use a username of  ag_te-testuser-1, then you would clearly violate the uniqueness, because all these attributes have already been used.
 +
* go to blueprint user
 +
[[File:Goto-testuser.png|none|300px|GTO Blueprint User]]
 +
* Copy blueprint user to newuser
 +
[[File:Copy-to-new-user.png|none|300px|copy to new user]]
 +
* Fill in attrinutes for new user
 +
[[File:New-entry-1.png|none|300px|attributes for new user]]
 +
[[File:New-entry-2.png|none|300px|attributes for new user]]
 +
[[File:New-entry-3.png|none|300px|attributes for new user]]
 +
 +
* Check on new user and submit
 +
[[File:Check-and-submit.png|none|300px|check and submit]]
 +
* OOps! Attribute (uid=9003 was not unique); Error message
 +
[[File:Attribute-not-unique.png|none|300px|Error Message]]
 +
* Make attribute unique (uid=9004)
 +
[[File:Change_for_success.png|none|300px|Error Message]]
 +
* Success!
 +
[[File:Success2.png|none|300px|Success]]
 +
 +
 +
* the researcher can now log in the web-interface and change her password.
 +
* the researcher can now ssh to hpc3, the according home directory will be created.
 +
* the researcher can transparently access the fat but a bit slower storage via
 +
cd /group.storage/ag_YOUR-GROUP

Latest revision as of 08:49, 2 October 2021

The High Performance Cluster of WiWi

Contents

[edit] 1 How To for users

  • Approach your group admin (see list below) to create an account for you, or change your password if you have forgotten your old one.
  • Your group admin will come back to you with your account (username and password). A user Mustermann's account (member of Arbeitsgruppe TestAG, ag_te) might be created as ag_te-mustermann.
  • From within the university's network (or VPN if working from home), the user Mustermann might use the command line to connect to the HPC:
ssh  ag_te-mustermann@hpc3.wiwi.univie.ac.at 

If a session is connected for the first time, just accept the fingerprint question and continue with your password. Your home directory will be created. Once logged in, your remote prompt will look like:

[ag_te-mustermannl@hpc3 ~]$

Now you can run your commands; e.g.

[ag_te-mustermannl@hpc3 ~]$ cd; pwd

that will return the name of your home directory.

If you would like to transfer files into your home-directory, you could use a sftp client (like WinSCP, Cyberduck, ...).

You should clear up once your computing is done. In case you want some files to be shared with your group members for later use, you could use /group.storage/ag_testAG; whereas ag_testAG has to be replaced with your ag_Arbeitsgruppe.

[ag_te-mustermannl@hpc3 ~]$  cd /group.storage/ag_testAG

ag_Arbeitsgruppe might be created on the fly so that you can change into it. Physically, you are at another, slower machine, which could be shown by

[ag_te-mustermann@hpc3 ag_testAG]$ df -h .
  Dateisystem                  Größe Benutzt Verf. Verw% Eingehängt auf
  172.16.1.101:/home/ag_testAG   91T    1,6T   90T    2% /group.storage/ag_testAG
  • How to run your jobs, see Slurm.


[edit] 2 How To for Group Admins

Groups are independently administered by their group admins. To indicate the group-spaces, the User-IDs (uid) are the Group IDs (gid) plus some addition.

I would suggest prefixing the users with ag_(+2 letters of group).

A new user in ag_hautsch should be ag_ha-USERXYZ. Her uid=7000+X, and her home-directory /home/ag_ha-USERXYZ.


[edit] 2.1 Specs

Gruppe Prof. Krivobokova: Hr. Stefan Geissler (stefan.geissler@univie.ac.at)
Gruppe Prof. Cuchiero: Hr. Guido Gazzani (guido.gazzani@univie.ac.at)
Gruppe Prof. Ehmke: Hr. Rico Kötschau (rico.koetschau@univie.ac.at)
Gruppe Prof. Dörner: Roland Braune, Georg Fröhlich (georg.erwin.adrian.froehlich@univie.ac.at)
Gruppe Operations Research: Michael Kahr (m.kahr@univie.ac.at)

[edit] 2.2 implementation

ag_testAG_ou        ag_testAG      (gid=9000) andreas.loibl@univie.ac.at                  uid=9000 + X 

                                          DN: uid=ag_ha-loibl,ou=ag_hautsch_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at
                                           DN: all the other admins as well
ag_krivobokova_ou   ag_krivobokova (gid=8000) stefan.geissler@univie.ac.at                uid=8000 + X

                                          DN: uid=ag_kr-geissler,ou=ag_krivobokova_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at
ag_cuchiero_ou      ag_cuchiero    (gid=3000) guido.gazzani@univie.ac.at                  uid=3000 + X   

                                          DN: uid=ag_cu-gazzani,ou=ag_cuchiero_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at
ag_ehmke_ou         ag_ehmke       (gid=4000) rico.koetschau@univie.ac.at                 uid=4000 + X

                                           DN: uid=ag_em-koetschau,ou=ag_ehmke_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at
ag_doerner_ou       ag_doerner     (gid=5000) georg.erwin.adrian.froehlich@univie.ac.at   uid=5000 + X

                                           DN: uid=ag_do-froehlich,ou=ag_doerner_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at

                                           roland.braune@univie.ac.at 
                                           DN: uid=ag_do-braune,ou=ag_doerner_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at
ag_operres_ou       ag_operres     (gid=6000) m.kahr@univie.ac.at                         uid=6000 + X

                                           DN: uid=ag_op-kahr,ou=ag_operres_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at
ag_hautsch_ou       ag_hautsch     (gid=7000) andreas.loibl@univie.ac.at                  uid=7000 + X

                                           DN: uid=ag_ha-loibl,ou=ag_hautsch_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at

In case any admin wants to use SSH (instead of the ldap web interface), only the uid would be necessary to log in (within the uni's network):

ssh ag_ha-loibl@hpc3.wiwi.univie.ac.at

[edit] 2.3 Step by Step

Group admin is within the university's network (or using VPN if working from home).

If you want to test a bit, ag_testAG_ou is the playground to do so.

uid=ag_ha-loibl,ou=ag_hautsch_ou,ou=People,dc=hpc,dc=wiwi,dc=univie,dc=ac,dc=at
password 12345
Login

If you might want to change your password, click your uid:

Change Password
  • Connect via ssh:
ssh ag_ha-loibl@hpc3.wiwi.univie.ac.at
  • Within this ssh session, Group admin can access OpenLdap directory entry (not the local /etc/passwd!)
getent passwd | grep ag-XY
SSH

If you assigned an uid of 9001, 9002, or 9003 to a new user, or /home/ag_te-testuser-1 as a home directory, or use a username of ag_te-testuser-1, then you would clearly violate the uniqueness, because all these attributes have already been used.

  • go to blueprint user
GTO Blueprint User
  • Copy blueprint user to newuser
copy to new user
  • Fill in attrinutes for new user
attributes for new user
attributes for new user
attributes for new user
  • Check on new user and submit
check and submit
  • OOps! Attribute (uid=9003 was not unique); Error message
Error Message
  • Make attribute unique (uid=9004)
Error Message
  • Success!
Success


  • the researcher can now log in the web-interface and change her password.
  • the researcher can now ssh to hpc3, the according home directory will be created.
  • the researcher can transparently access the fat but a bit slower storage via
cd /group.storage/ag_YOUR-GROUP
Personal tools
Namespaces

Variants
Actions
Navigation
Tools