Differences

This shows you the differences between two versions of the page.

--- section:algosec:documentation:usefullcommands [2021/11/20 19:58] – patrik
+++ section:algosec:documentation:usefullcommands [2023/09/29 07:01] (current) – external edit 127.0.0.1
@@ Line 3: / Line 3: @@
 ====== Useful Algosec commands, troubleshooting and information ======
+//
+by Patrik Hermansson
+//
 ===== Usefull commands, summary =====
   * [[usefullcommands#Regarding passwords|Regarding passwords]]
@@ Line 51: / Line 54: @@
 ===== Other useful information, summary =====
-  * Needed ports for cluster and functions
+  * [[usefullcommands#Needed ports for cluster and functions|Needed ports for cluster and functions]]
-  * Where the risk profiles are located
+  * [[usefullcommands#Where the risk profiles are located|Where the risk profiles are located]]
-  * Regex for search in Notepad++
+  * [[usefullcommands#Regex for search in Notepad++|Regex for search in Notepad++]]
-  * How to use screens in Linux
+  * [[usefullcommands#How to use screens in Linux|How to use screens in Linux]]
-  * How to identify users in the system
+  * [[usefullcommands#BZIP2 and GZIP archiving|BZIP2 and GZIP archiving]]
-  * BZIP2 and GZIP archiving
+  * [[usefullcommands#TAR commands|TAR commands]]
-  * TAR commands
+  * [[usefullcommands#How to Encrypt and Decrypt Files and Directories Using Tar and OpenSSL|How to Encrypt and Decrypt Files and Directories Using Tar and OpenSSL]]
-  * How to Encrypt and Decrypt Files and Directories Using Tar and OpenSSL
+  * [[usefullcommands#How to activate debug mode in ABF|How to activate debug mode in ABF]]
-  * How to activate debug mode in ABF
+  * [[usefullcommands#Boostmode on and off|Boostmode on and off]]
-  * Boostmode on and off
+  * [[usefullcommands#To get destination NAT from firewalls in ASMS databas|To get destination NAT from firewalls in ASMS databas]]
-  * To get destination NAT from firewalls in ASMS databas
+  * [[usefullcommands#Cluster node suddanly removed from cluster|Cluster node suddanly removed from cluster]]
-  * Cluster node suddanly removed from cluster
+  * [[usefullcommands#How the user field in ABF flows work|How the user field in ABF flows work]]
-  * How the user field in ABF flows work
+  * [[usefullcommands#How to get a session id|How to get a session id]]
-  * How to get a session id
+  * [[usefullcommands#How to look into .tar, .zip, .bz2 files without unpacking them|How to look into .tar, .zip, .bz2 files without unpacking them]]
-  * How to look into .tar, .zip, .bz2 files without unpacking them
+  * [[usefullcommands#How to clean up the session database table in postgres|How to clean up the session database table in postgres]]
-  * How to clean up the session database table in postgres
+  * [[usefullcommands#Guide for LVM on new setup virtual appliance|Guide for LVM on new setup virtual appliance]]
-  * Guide for LVM on new setup virtual appliance
@@ Line 656: / Line 658: @@
 Ended up deleting the account with upper case first letter and user could login again after that.
-//
+=== Needed ports for cluster and functions ===
-by Patrik Hermansson
+If possible for uptime allow any ports between the nodes in HA. \\
+Twice i have had problem at customer due to the fact Algosec did not update there documentation for needed ports. \\
+Not in documentation nor in KB. (Custemer have PVLAN with ACL) \\
+^Type		^Port	^CM ↔ Slave	^CM ↔ RA	^Slave ↔ Slave	^HA/DR(2018.2)  ^HA/DR (2018.1)	^HA/DR (2017.3) ^
+|icmp 		 | V 		   | V 	  | - 		   | V           | V 		  | V |
+|ssh 		|tcp/22 	 | V 		  | V 	  | - 		     | V           | V 		  | V |
+|https 	| tcp/443    | V 		   | V 	  | - 		     | V            | - 		| - |
+|syslog |udp/514   | - 		  | - 	  | - 	   	    | V           | -		       | - |
+|hazelcast |tcp/5701   | V 		  | - 	  | V 		    | V           | V 		  | - |
+|activemq 	|tcp/61616 | V 		  | - 	  | - 		    | V           | - 		  | - |
+|postgresql |tcp/5432  | V 		  | - 	  | - 	   	    | V           | V 		  | - |
+|pgpool    |tcp/5433   | V        | -     | -           | V           | -         | - |
+|HA/DR 	  |tcp/9595    | - 		  | - 	  | - 		    | V           | V 		  | - |
+|heartbeat	|udp/694   | - 		  | - 	  | - 		    | -           | - 		  | V |
+(not in use since 2018.1)
+\\
+\\
+\\
+Ports required for communications (central manager, remote agents) in geo-distributed architecture.\\
+. For configuration procedures (adding a remote agent, adding/editing/deleting devices) that must be synchronous:\\
+^Port 	^Protocol 	^Description 	^Purpose ^
+|22 	|TCP 	|SSH 	|Required for running commands upon the remote agent from the central manager |
+. For log collection, monitor and data collections procedures that may be asynchronous:\\
+^Port 	^Protocol 	^Description 	^Purpose ^
+|443  	|TCP 	     |SOAP over HTTPS 	|Required for running commands and obtaining the status of the remote agent and current actions performed on it|
+|22 	     |TCP 	     |SCP 	|Required for copying files to and from the remote agent|
+. For communications between master-slave in load-distributed architecture:\\
+^Port 	^Protocol 	^Description 	^Purpose ^
+|443 	     |TCP 	     |SOAP over HTTPS 	|From master to slave|
+|22 	     |TCP 	     |SCP-SSH 	     |From master to slave|
+|5432 	|TCP 	     |Postgresql |
+|5433      |TCP       |PGPool  	 |
+=== Where the risk profiles are located ===
+/home/afa/.fa/risk_profiles/ \\
+  Files are saved as *.xml
+=== Regex for search in Notepad++ ===
+To search after ip-addresses in Notepad++ use the following regex:
+  [0-9]+.[0-9]+.[0-9]+.[0-9]+
+  or
+  \b(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(?1)){3}\b
+=== How to use screens in Linux ===
+Screen(s) is a good way to make sure that updates or other things will keep on going and that infromation in the shell will be avialiable even if the connection is severed between the client and the server. \\
+You can reconnect to the session and the information is still there. \\
+\\
+For the Algosec session i will use the session-name of algosec \\
+\\
+#Start a screen session
+  screen
+OR
+  screen -S [name of session]
+#List active screens on the system
+  screen -ls
+OR
+  screen -list
+  --- *** output *** ---
+  #named session algosec
+  [root@server ~]# screen -ls
+  There is a screen on:
+.algosec   (Detached)
+Socket in /var/run/screen/S-root.
+#Connect to an detached screen on the system
+  screen -r [screenname/session]
+  screen -r algosec
+=== BZIP2 and GZIP archiving ===
+This can be done with singel file(s) \\
+* GZIP * \\
+Compress a single file \\
+  gzip [filename] ## This will create a compressed file and remove the original file.
+Compress multiple files at once \\
+  gzip [filename1] [filename2] [filename3]
+Compress a single file and keep the original \\
+  gzip -c [filename] > [filename].gz
+Decompress a gzip compressed file \\
+  gzip -d [filename]
+  or
+  gunzip [filename]
+Decompress a gzip file but keep the original compressed file \\
+  gunzip -c [filename].gz > [filename]
+* BZIP2 * \\
+Create archive \\
+  bzip2 [filename] ## This will create a compressed file and remove the original file.
+If not to delete the original file use -k \\
+  bzip2 -k [filename] ## The original file is NOT deleted
+Decompress archives \\
+  bzip2 -d [filename]
+=== TAR commands ===
+To build tar archives \\
+  tar -vcf [filename]
+To build tar archives with gzip encryption \\
+This can be done with folder(s) \\
+  tar -zvcf [filename.tar.gz] [file/folder1] [file/folder2] [...]
+To build tar archives with bzip2 encryption \\
+This can be done with folder(s) \\
+bzip2 have a harder compression, more CPU demanding \\
+  tar -jvcf [filename.bz2] [file/folder1] [file/folder2] [...]
+To Decompress tar archive \\
+  tar -xvf [filename]
+To Decompress tar archive with gzip \\
+  tar -zxvf [filename].tar.gz
+To Decompress tar archive with bzip2 \\
+  tar -jxvf [filename].bz2
+=== How to Encrypt and Decrypt Files and Directories Using Tar and OpenSSL ===
+Remember that the conventional form of using OpenSSL is: \\
+openssl command command-options arguments \\
+To encrypt the contents of the current working directory \\
+  tar -czf - * | openssl enc -e -aes256 -out secured.tar.gz
+  Explanation of the above command:
+  enc – openssl command to encode with ciphers
+   -e – a enc command option to encrypt the input file, which in this case is the output of the tar command
+   -aes256 – the encryption cipher
+   -out – enc option used to specify the name of the out filename, secured.tar.gz
+\\
+\\
+Decrypt Files in Linux \\
+To decrypt a tar archive contents, use the following command. \\
+  openssl enc -d -aes256 -in secured.tar.gz | tar xz -C test
+  Explanation of the above command:
+   -d – used to decrypt the files
+   -C – extract in subdirectory named test
+=== How to activate debug mode in ABF ===
+You need to enable debug mode to troubleshoot BusinessFlow. \\
+Solution\\
+To enable debug mode in BusinessFlow: \\
+. Log in to AFA as root user using SSH.
+. Edit the following file /home/bflow/config/log4j2.xml:
+    a. Change the following line:
+    <property name="algosec-log-level">INFO</property>
+    to
+    <property name="algosec-log-level">DEBUG</property>
+  (( 3. Restart the apache-tomcat service. )) ###usually not needed
+=== Boostmode on and off ===
+Script download: https://algosec.sharefile.com/d-s62142b58f5b4210b \\
+\\
+\\
+To install boostmode, perform the following, unzip the script and move it to the system under /tmp/ (this can be applied on GEO’s, Slaves, HA secondaries, all necessary boxes). \\
+\\
+As root:
+  cp /tmp/boostmode /etc/init.d/boostmode
+  chmod 755 /etc/init.d/boostmode
+  chkconfig boostmode on
+  service boostmode start # this may take a few minutes
+After the service has started, you must restart all the relevant services:
+  service activemq restart
+  service apache-tomcat restart
+  service algosec-ms restart
+  service postgresql reload
+  restart_fireflow
+It’s important to communicate to the customer that they will also need to perform the following steps after applying any hotfixes or patches in the future – as these can overwrite some of the boostmode settings. \\
+After successfully installing hotfixes the following should be run as root:\\
+  service boostmode start
+  service activemq restart
+  service apache-tomcat restart
+  service algosec-ms restart
+  service postgresql reload
+  restart_fireflow
+Boost mode can be disabled just by running:
+  service boostmode stop
+Stopping the service will roll back all the changes. \\
+Before
+  -------
+  [root@algosec-RA ~]# swapon -s
+  Filename                                Type            Size    Used    Priority
+  /dev/dm-1                               partition       7688188 0       -1
+  [root@algosec-RA ~]#
+After
+  ------
+  [root@algosec ~]# swapon -s
+  Filename                                Type            Size    Used    Priority
+  /dev/zram0                              partition       3087552 0       100
+  /dev/zram1                              partition       3087552 0       100
+  /dev/zram2                              partition       3087552 0       100
+  /dev/zram3                              partition       3087552 0       100
+  [root@algosec ~]#
+=== Cluster node suddanly removed from cluster ===
+Problem: \\
+On node in the cluster removed from the cluster. \\
+\\
+Logs: \\
+HA logs (/var/log/algosec_hadr/ several logs i this location) \\
+Messages log (/var/log/message) \\
+\\
+Low disk space: \\
+On one node, in ha logs (collect from ha menue (algosec_conf 13)). Or in HA logs, there will be a entry of low disk space and that cluster will be broken due to that.  Log of this is on the node that was removed from the cluster. \\
+To low disk space is less that 10% free space (on any partition? Maby but surely on the /data partition)\\
+=== How the user field in ABF flows work ===
+The field is populated from either the ABF database OR the supported firewall. (like Palo Alto/Panorama). \\
+You cannot combine the two source of users, like rest of the produkt. Only one or the other is used at one or the other time. \\
+Also the users possibly to populate the user field needs to be present in the firewall. More test on this should be done to verify. \\
+Settings to change this is found uder:\\
+ABF => [name in upper right corner] => Administration => Configuration => User Awareness Support => USer validation via LDAP is Currently [on/off] \\
+\\
+If on = get from firewall \\
+If off = get from ABF user database \\
+=== How to get a session id ===
+GUI:
+. Go to the AFA home page (the portion displaying graphs).
+. In the Web browser box, type ?"!session!" .
+. Click Enter.
+  A popup displays a unique session ID.
+CLI:
+. Go to the CLI and type the following command:ls -ltr /home/afa/public_html/algosec | tail
+  A list of session IDs displays.
+. Make a note of the latest session ID.
+=== How to look into .tar, .zip, .bz2 files without unpacking them ===
+.tar
+.zip
+  zcat [cat]
+  zmore [more]
+  zless [less]
+Or if the .zip contains multiple files
+  vim [file].zip
+Example
+  zcat testfile.zip
+  vim testfile.zip
+*** .bz2
+  bzcat [cat]
+  bzless [less]
+  vim
+=== How to clean up the session database table in postgres ===
+In some versions of ASMS the session table just grows and grows. This is a bug! \\
+To manually empty the database session table do the procedure below. \\
+  #########################
+  # Important before synk #
+  #########################
+  ++++++++++++++++++++++++++++++++++++++++
+  + Check the postgres /session db table +
+  ++++++++++++++++++++++++++++++++++++++++
+This is the procedure \\
+\\
+On the active node (where all services are runnig AFA,AFF,DB) \\
+Stop services as follows: \\
+  /usr/share/fireflow/local/sbin/stop_fireflow.sh
+  service crond stop
+  service apache-tomcat stop
+  service algosec-ms stop
+  service postgresql stop
+  service activemq stop
+  service httpd stop
+  service logstash stop
+  service elasticsearch stop
+  service kibana stop
+  service mongod stop
+  service aff-boot stop
+Once all services are stopped bring the postgresql service back up with 'service postgresql restart' \\
+Once postgres run the following commands from the CLI. \\
+  psql -U postgres -d rt3 -c 'truncate sessions;'
+  psql -U postgres -d rt3 -c 'vacuum full verbose sessions;'
+Once the commands finish bring the rest of the services back online. \\
+  service crond start
+  service httpd start
+  service postgresql start
+  service activemq start
+  service apache-tomcat start
+  service algosec-ms start
+  service aff-boot start
+  /usr/share/fireflow/local/sbin/start_fireflow.sh
+  service logstash start
+  service elasticsearch start
+  service kibana start
+  service mongod start
+=== Guide for LVM on new setup virtual appliance ===
+  ################
+  # Up to A30.20 #
+  ################
+Fix the LVM on the devices \\
+Lists all disks in the system (as fdisk -l) \\
+  lsblk
+  parted /dev/sdb
+  moves from msdos to guided partition table for disks over 2TB
+  mktable GPT
+  Creates a partition of 50GB
+  mkpart 0 1 50000
+  Create a partition of the rest of the disk
+  mkpart 0 50001 100%
+  Lists all disks in the system (as fdisk -l)
+  lsblk
+  Creates the physical volumes of the new partitions
+  pvcreate /dev/sdb1
+  pvcreate /dev/sdb2
+  Extend the volume group /dev/vg_algsoec with the new physical volumes
+  vgextend /dev/vg_algosec /dev/sdb1
+  vgextend /dev/vg_algosec /dev/sdb2
+  Extend the logial volumes with the new partitions (-r will extend automaticly)
+  lvextend -r /dev/vg_algosec/vg_system /dev/sdb1
+  lvextend -r /dev/vg_algosec/vg_data /dev/sdb2
+  if not -r extends the logical volumes automaticly do the following
+  for ext4 filesystem
+  resize2fs /dev/vg_algosec/vg_system
+  resize2fs /dev/vg_algosec/vg_data
+  for xfs filesystem
+  xfs_growfs /dev/vg_algosec/vg_system
+  xfs_growfs /dev/vg_algosec/vg_data
+  To check the filesystem expends ok via watch per second
+  screen
+  watch -n 1 -d "df -hT"
+  ip addr
+  watch -n 1 -d "df -hT"
+  ###################
+  # For ASMS V32 => #
+  ###################
+Differences is that volume group and logical volumes have new names / different locations. \\
+\\
+Fix the LVM on the devices\\
+Lists all disks in the system (as fdisk -l)\\
+  lsblk
+  parted /dev/sdb
+  moves from msdos to guided partition table for disks over 2TB
+  mktable GPT
+  Creates a partition of 50GB
+  mkpart 0 1 50000
+  Create a partition of the rest of the disk
+  mkpart 0 50001 100%
+  Lists all disks in the system (as fdisk -l)
+  lsblk
+  Creates the physical volumes of the new partitions
+  pvcreate /dev/sdb1
+  pvcreate /dev/sdb2
+  Extend the volume group /dev/vg_algsoec with the new physical volumes
+  vgextend /dev/centos /dev/sdb1
+  vgextend /dev/centos /dev/sdb2
+  Extend the logial volumes with the new partitions (-r will extend automaticly)
+  lvextend -r /dev/centos/root /dev/sdb1
+  lvextend -r /dev/centos/data /dev/sdb2
+  if not -r extends the logical volumes automaticly do the following
+  for ext4 filesystem
+  resize2fs /dev/vg_algosec/vg_system
+  resize2fs /dev/vg_algosec/vg_data
+  for xfs filesystem
+  xfs_growfs /dev/vg_algosec/vg_system
+  xfs_growfs /dev/vg_algosec/vg_data
+  To check the filesystem expends ok via watch per second
+  screen
+  watch -n 1 -d "df -hT"
+  ip addr
+  watch -n 1 -d "df -hT"