Linux常用命令集合

这篇博客记录一些我比较有用但是的command, 方便自己查询, 标题加入了一些英语翻译，方便CTRL+F。 欢迎补充

网络(Network related)

1. curl查询公网出口IP（use curl to query public IP）

   curl ifconfig.me

   some other useful webstite:

   1. ip.cn
   2. ipinfo.io
   3. cip.cc
   4. myip.ipip.net

2. 设置ssh反向tunnel(ssh reverse tunnel)

   A机器在内网，B机器在外网，C机器可以连接B但是不能连接A。如果C想通过B链接到A机器

   1. 从A登录B机器
      ssh -fNR 8771:localhost:22 [email protected]_MACHINE_IP
   2. 这样C机器就可以访问 ssh -p 8771 [email protected]_MACHINE_IP

3. 列出监听端口(list all listening port)

   netstat -anlp | grep -w LISTEN

   or

   sudo netstat -tulp

4. 打开关闭网卡(up and down network interface)

   ip link set eth0 up/down

5. 用ssh执行远程命令(use ssh to execute remote commands)

   -t Force pseudo-terminal allocation. This can be used to execute arbitrary screen-based programs on a remote machine, which can be very useful, e.g. when implementing menu services. Multiple -t options force tty allocation, even if ssh has no local tty.

   ssh -t [email protected] "command1 & command2"

   For example, if you want to simply pull a docker images on multiple servers, you can write a simple bash script like this:

   for remote in 192.168.1.1 192.168.1.2 192.168.1.3; do
       ssh -t [email protected] "docker pull image:tag"
   done

   if you want to execute remote command in background, you can try to use nohup or screen:

   ssh [email protected] screen -d -m ./script

   and you can check the result with

   screen -ls 

6. Show network interface RX-OK/RX-ERROR

   ip addr show;echo;echo;ip route list table all;echo;echo;netstat -ai

   文件查找和字符操作(counting, searching and string operation)

7. grep查找包含特定文字的文件(use grep to find all files container specific text)

   grep -rnw '/path/to/somewhere/' -e 'pattern'
   
     -r or -R is recursive,
     -n is line number, and
     -w stands for match the whole word.
     -l (lower-case L) can be added to just give the file name of matching     files.
   
   Along with these, --exclude, --include, --exclude-dir flags could be     used for efficient searching:
   
     This will only search through those files which have .c or .h     extensions:
   
     grep --include=\*.{c,h} -rnw '/path/to/somewhere/' -e "pattern"
   
     This will exclude searching all the files ending with .o extension:
   
     grep --exclude=*.o -rnw '/path/to/somewhere/' -e "pattern"
   
     For directories it's possible to exclude a particular directory(ies)     through --exclude-dir parameter. For example, this will exclude the     dirs dir1/, dir2/ and all of them matching *.dst/:
   
     grep --exclude-dir={dir1,dir2,*.dst} -rnw '/path/to/somewhere/' -e     "pattern"

8. grep从文件中过滤并统计特定字符数量(use grep to filter string in file and count number)

   grep -n '2018-03-24' /var/log/xxx/xxx/xxx.log  | grep 'stringyouwant'  |     wc -l

9. find查找并替换文件中内容(use find to filter string and replace)
   stackoverflow

   find . -iname '*.txt' -exec sed -i 's/text-to-search/text-to-replace/g' {} \;

10. find查找过滤文件中内容(use find to filter string from file)

    find . -iname '*.txt' | xargs -n 1 grep -nR 'text-to-search'

11. find recursively find all files in current and subfolders based on wildcard matching

    find . -name "foo*"

12. find exclude a directory

    1. use -prune switch

       find . -path ./misc -prune -o -name '*.txt' -print

    2. use -not -path

       find -name "*.js*" -not -path "./directory/*"

       or

       find -name "*.js*" -not -path "./directory*"

13. find and do operations(like copy)

    find . -name "*.dae" -not -path "./ignore-directory/*" -exec cp {} ../otherfolder/ \;

    better(not tested)

    find . -name '*.dae' -not -path './ignore-directory/*' -exec cp {} ../otherfolder/ +

14. 查看当前目录下多少文件(Counting Files in the Current Directory

    ls -1 | wc -l
    ls -l | grep -v ^l | wc -l (exclude symbol file, that's an "L" not a "1" this time) 

15. 遍历当前目录下所有文件名(iterate over filenames in current directory)

    for filename in echo *; do
        echo "Processing $filename"
        # do something on $f
        done

16. 删除字符串后缀(Remove the string prefix)
    Shell-Parameter_Expansion

    $ x="/foo/fizzbuzz.bar"
    $ y=${x%.bar}
    $ echo ${y##*/}
    fizzbuzz

    Shell

17. 查看当前shell版本(show which shell are you using)

    echo $0

    Debian/Ubuntu System related

18. 如何压缩一个文件名是--开头的文件?
    如果遇到一个文件名是以--开头的,那么在linux命令行中会被认为是参数. 例如文件名--test_abc

    tar cvf test.tar.gz --test_abc

    这时候只要用--来表示参数终结就可以了

    tar cvf test.tar.gz -- --test.abc

19. Showing Linux Kernel LOG/Query the system journal

    sudo dmesg

    or use journalctl under debian/ubuntu

    1. all dmesg output in the last 2 hours

       sudo journalctl -k 

    2. all journal since last boot

       sudo journalctl -b

    3. list boots in the journal

       journalctl --list-boots

20. Setting persistent logging for kernel log

    by default the log is written non-persistently to /run/systemd/journal/(a binary file)

    Edit the /etc/systemd/journald.conf and uncomment #Storage=auto and change auto to persistent

    Then you can restart the service

    sudo systemctl restart system-journald

21. Make a read-only filesystem writable

    remouting it read-write:

    sudo mount -o remount,rw '/mount-point'

22. Alternative different version of software

    If you have installed multiple versions of gcc, like gcc-6 and gcc-9. your /usr/bin/gcc is probably pointing to gcc-6 by default. You can use the following command to change the version.

    sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 900

    open the windows for switching version

    sudo update-alternatives --config gcc

Emacs

1. compile .emacs.d directory

   C-u 0 M-x byte-recompile-directory

   will compile all the .el files in the directory and in all subdirectories below.
   The C-u 0 part is to make it not ask about every .el file that does not have a .elc counterpart.

   From stackoverflow

2. upgrade emacs packages

   M-x list-packages

   1. press U to mark all upgradable packages to be upgrade
   2. press x to perform the new updates

2. format code

   select the region you want to format

   M-h

   M-x (indent-region)

   see also : choosing mode

3. Regex in emacs

Browser

1. open recent closed tab (chrome)

   There are 3 ways to do it:

   1. Get it back by right-clicking in the tab bar and selecting Reopen closed tab from the menu
   2. By clicking Ctrl+Shift+T.
   3. You can also find a list of recently closed tabs in the Settings menu. Click on the 3 horizontal line icon top right and choose Recent tabs.

Other commands and tools

screen

1. killing all the detached screens    

screen -ls | grep detached | cut -d. -f1 | awk '{print $1}' | xargs kill

or inside screen

Ctrl-a \

Unclutter

Unclutter hides your X mouse cursor when you do not need it, to prevent it from getting in the way. You have only to move the mouse to restore the mouse cursor. Unclutter is very useful in tiling window managers where you do not need the mouse often.

Extract GPS Info from Photos (Python脚本提取照片中的GPS信息)

1. pip install pillow

2. run this code under the folder image

   from PIL import Image
   from PIL.ExifTags import TAGS
   import glob
   
   
   types = ('*.JPG',)
   
   def demo(name):
       exifdata = {}
       imgfile = Image.open(name)
       info = imgfile._getexif()
       if info:
           for (tag, value) in info.items():
               # print(str(tag) + ' : ' + str(value))
               decoded = TAGS.get(tag, tag)
               # print str(decoded)
               exifdata[decoded] = value
           exifgps = exifdata.get('GPSInfo', '')
           if exifgps:
               print('it has founded ; ' + str(name) + "  " + str(exifgps))
   def main():
       for t in types:
           filenames = glob.glob(t)
           [demo(f) for f in filenames]
   if __name__=='__main__':
       main()

3. If you have multiple image types, you can change types tuple to add more, for exmaple, types = ('\*.JPG', '*.png')