文件与文件夹操作 文件夹操作 ls # 查看文件夹 ls your_dir ls -F # 显示文件类型，/:文件夹，*:可执行文件，@:链接 ls -F your_dir | grep / # 只显示文件夹 ls -F your_dir | grep -v / # 只显示文件 tree # 以树形显示文件夹 tree tree -L 2 # 参看目录层数2以内的文件 tree -f # 显示完整路径 tree -d # 只显示文件夹 pwd # 显示当前目录 pwd mkdir # 创建文件夹 mkdir dir mkdir -p your_dir/subdir # 创建多级文件夹 tree -fid --noreport your_dir >> your_dir.txt && mkdir tmp && cd tmp/ && mkdir -p `cat ../your_dir.txt` # 复制目录结构 touch # 创建文件 touch your_file # 创建文件或更新文件时间戳 touch -a|-m your_file # 只更新访问|修改时间 touch -t 202102071200.00|YYYYmmddHHMM.SS # 更新为指定时间戳 cd # 切换文件夹 cd your_dir cd ..|~|- # 切换到父文件夹|用户文件夹|上一次文件夹 cp # 复制文件夹 cp source target cp -p # 保留源文件所有属性 cp -d # 保留符号链接指向 cp -r # 递归复制文件夹 cp -a # == cp -pdr mv # 移动文件夹 mv source target mv source_1 source_2 source_3 target # 移动多个文件到目标文件夹 rm # 删除 rm file rm -rf dir # 你懂的 rename # 重命名文件 rename .png .jpg ./* # 将所有文件的后缀由 .png 改成 .jpg ln # 链接 ln your_target your_link # hard link，多个文件名指向同一个索引节点（inode），只有当文件所有的硬链接都被删除时，文件才真正删除。可以通过创建硬链接防止文件误删 ln -s your_target your_link # soft/symbolic link，相当于创建快捷方式 # 查看链接具体的命令 type ll # 输出：ll is aliased to `ls -l --color=auto' 文件夹路径 basename # 显示文件名 basename /data/dir1/file1.txt basename /data/dir1/file1.txt .txt # 不显示后缀 .txt dirname # 显示路径 dirname /data/dir1/file1.txt dirname ./file1.txt # 传入相对路径也会返回相对路径 查看文件信息 ls -l 输出的含义 1|2 |3 |4 |5| 6 | 7 | 8 | 9 | 10 -rw-rw-r-- 1 yzchen yzchen 6 Feb 7 15:23 newfile drwxrwxr-x 2 yzchen yzchen 4096 Feb 7 15:25 subdir 文件（夹）类型 文件所属用户权限 文件所属用户组权限 其他用户权限 硬链接个数 所属用户 所属用户组 文件（夹）大小 修改时间 名称 file 查看文件类型 ...

Face Detection retina face Github - retinaface: deep learning based cutting-edge facial detector deepface Github - deepface: a lightweight face recognition and facial attribute analysis (age, gender, emotion and race) framework Install: pip install deepface Usage: from deepface import DeepFace # Detection: 从照片中检测人脸 face_objs = DeepFace.extract_faces( img_path = "img.jpg", detector_backend = 'retinaface', # retina face 检测效果较好 align = False, enforce_detection = False, ) # Verification: 判断两张图片是否同一个人 result = DeepFace.verify(img1_path = "img1.jpg", img2_path = "img2.jpg") # 默认会先做 detection，如果是已经提取了人脸的照片可以用 result = DeepFace.verify(img1_path = "img1.jpg", img2_path = "img2.jpg", detector_backend='skip') # 指定相似度阈值 result = DeepFace.verify(img1_path = "img1.jpg", img2_path = "img2.jpg", threshold=0.5) # Embedding: 提取人脸 embedding，可以用于后续的对比计算 embedding_objs = DeepFace.represent(img_path = "img.jpg") # Recognition: 人脸识别，从人脸库中找出最符合的人 dfs = DeepFace.find( img_path = "img1.jpg", db_path = "./workspace/my_db", model_name = "VGG-Face", # 默认模型是 VGG-Face ) # Analysis：提取脸部特征（年龄、性别、情绪等） objs = DeepFace.analyze( img_path = "img4.jpg", actions = ['age', 'gender', 'race', 'emotion'], )

Crypto Exchanges coinmarketcap Binance okex 火币 coinbase coinbase pro gemini kraken bitstamp swap cash.app Wallet Electrum https://electrum.readthedocs.io/en/latest/ seed：squeeze harsh aware aisle lemon blood dog paddle size super reflect two Google Authenticator biance：PATFWTFTREMYRFZT Tools CCXT Install pip install ccxt Usage import ccxt print(ccxt.exchanges) Keywords sandwich attack MEV(Miner Extractable Value) Data Bitcoin Futures Market Breakdown coinglass laevitas Ref https://ethereum.org/en/developers/docs/ https://www.picol.com/zh-CN https://www.pionex.com/en-US Terms Spot：现货 Margin：杠杠 Derivatives Options expires Futures expires Perpetual Swaps：永续合约 no expiry date ...

SelectDataset

Snippets text center <div class="text-center"> <h1>Welcome</h1> </div> Design Margin & padding Flex A Complete Guide to Flexbox FLexbox Playground Fonts Google Fonts Top 10 Web Fonts Serif Georgia Times New Roman Merriweather Sans-Serif Roboto Open Sans Source Sans Pro Lato Raleway Poppins Ariel Color Color Harmonies Complementary colors: Colors that are directly opposite one another on the color wheel are known as complementary colors. Complementary colors have a high contrast and can be very effective as accent colors when paired with a more neutral palette. ...

Design Patterns UML类图 参考：设计模式之 UML 类图 UML例子 class Student { private String name; public String getName() { return name; } public void takeExam(Course course) { course.test(); } } class Course { private String courseName; public void test() { // take exam... } } 类名叫做 Student 和 Course + 代表 public 公共，- 代表 private 私有，# 代表 protected 成员变量类型写在前，参数名称写在后 函数传递参数，参数名写在前，类型写在后 函数返回值写在函数签名的后面 两个类之间若存在关系，可使用箭头进行关联，具体关联规则在下文介绍 箭头上的数字代表 1 个学生可以不参加课程，也可以无限制参加各种课程 1 代表一个，0..* 代表 0 个到无限个 类与类的关系 依赖 (Dependency) 对于两个相对独立的对象，当一个对象负责构造另一个对象的实例，或者依赖另一个对象的服务时，这两个对象之间主要体现为依赖关系。 ...

Docker Command 登录 docker login 查看信息 # version docker version docker info # 查看系统信息 docker system info # ps docker ps # 查看容器信息 docker inspect container_name/container_id # 查看容器运行日志 docker logs container_name/container_id # 查看尾部日志并跟踪 docker logs --tail 10 -f container_name/container_id # 查看容器端口映射 docker port container_name # 查看容器占用资源 docker stats 镜像 # search images docker search python # list images docker image ls # download image docker pull python:latest # commit image docker commit -m "comment" -a "author" container_id my_hub/my_image:tag # push image docker login docker tag my_image my_hub/my_image docker push my_hub/my_image # remove image docker rmi my_image:tag docker image prune # save image docker save -o my_image.tar my_hub/my_image # load image docker load --input my_image.tar Build docker image build ...

ELK-Elasticsearch Run on Docker docker pull docker.elastic.co/elasticsearch/elasticsearch:7.9.1 docker run -d -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" --name elasticsearch docker.elastic.co/elasticsearch/elasticsearch:7.9.1 常用查询 # 查看集群健康 curl "http://localhost:9200/_cat/health?v" # 查看节点 curl "http://localhost:9200/_cat/nodes?v" # 查看index curl "http://localhost:9200/_cat/indices?v&s=index" # 查看index按大小排序 curl "http://localhost:9200/_cat/indices?v&s=store.size:desc" # 查看数据，按照field1和field2的值过滤 curl "http://localhost:9200/my-index/_search?q=field1:100&q=field2:abc" # 查看数据，返回field1和field2并按field1排序 curl "http://localhost:9200/my-index/_search?_source=field1,field2&sort=field1:desc" # POST curl -XPOST "http://localhost:9200/my-index/_search?" -H 'Content-Type: application/json' -d '{"size":3, "query": {"term":{"Name":"Tom"}}}' # 查看配置 curl "http://localhost:9200/_cluster/settings?include_defaults=true&pretty" # 查看统计 curl "http://localhost:9200/_nodes/stats?pretty" APIs cat APIs cat APIs Common # verbose，显示 header curl -XGET "http://{ip}:9200/_cat/master?v" # help，显示说明 # 在 Kibana 中可以输入命令后按 `cmd + /` 查询文档 curl -XGET "http://{ip}:9200/_cat/master?help" # 输出指定 headers curl -XGET "http://{ip}:9200/_cat/master?h=ip,port,name" # 排序 curl -XGET "http://{ip}:9200/_cat/indices?v&s=index curl -XGET "http://{ip}:9200/_cat/indices?v&s=index:asc" # 升序 curl -XGET "http://{ip}:9200/_cat/indices?v&s=index:desc" # 降序 curl -XGET "http://{ip}:9200/_cat/templates?v&s=order:desc,index_patterns" # 多级排序 # 输出格式：format=text|json|yaml|smile|cbors curl -XGET "http://{ip}:9200/_cat/indices?format=json&pretty" # 输出单位：bytes=b|kb|mb|gb|tb|pb, curl -XGET "http://{ip}:9200/_cat/indices?s=store.size:desc&bytes=b" APIs ...

ELK-Filebeat Run Filebeat on Docker docker pull docker.elastic.co/beats/filebeat:7.9.1 docker run -d \ --name=filebeat \ --user=root \ --volume="$(pwd)/filebeat.docker.yml:/usr/share/filebeat/filebeat.yml:ro" \ --volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" \ --volume="/var/run/docker.sock:/var/run/docker.sock:ro" \ --volume="/var/log:/var/log" \ # mount host log position docker.elastic.co/beats/filebeat:7.9.1 filebeat -e -strict.perms=false Filebeat Config # file input filebeat.inputs: - type: log enabled: true paths: - /var/log/test.log exclude_lines: ['^nohup'] # modules filebeat.config: modules: path: ${path.config}/modules.d/*.yml reload.enabled: false # logstash output output.logstash: hosts: ["localhost:5044"] # elasticsearch output output.elasticsearch: hosts: ["localhost:9200"] # 当ES在docker中运行时，要用host ip而不能写localhost，因为Filebeat和ES不在一个docker中 username: 'username' password: 'password'

ELK-Logstash Run Logstash on Docker docker pull docker.elastic.co/logstash/logstash-oss:7.9.1 docker run --rm -it \ -p 5044:5044 \ -v /data/yzchen/elk/logstash/pipeline/:/usr/share/logstash/pipeline/ \ docker.elastic.co/logstash/logstash-oss:7.9.1 Logstash Config input { # Filebeat input beats { port => "5044" } # Redis input redis { host => "host_ip" port => port password => "password" data_type => "channel" key => "channel_pattern" } filter { # drop data if [type] != "raw" { drop { } } # parse json string json { source => "msgdata" } # drop field prune { blacklist_names => ["msgdata"] } # check if a field exists if ([foo]) { } # for numeric type if ("" in [foo]) { } # for other type # dissect dissect { mapping => { "line" => "%{month} %{day} %{time} %{id} %{?NET_CONNTRACK->}[%{type}]%{rest}" } } # select from array and dict mutate { update => {"new_field" => "%{[array_name][0][key_name]}"} } # qeury mysql and add filed jdbc_streaming { jdbc_driver_library => "/usr/share/logstash/mysql-connector-java-5.1.34.jar" jdbc_driver_class => "com.mysql.jdbc.Driver" jdbc_connection_string => "jdbc:mysql://host:3306/database" jdbc_user => "username" jdbc_password => "secret" statement => "SELECT col FROM table WHERE name = :name" parameters => { "name" => "name"} target => "col" } # query elasticsearch and add filed # 注意这里的query语法是ES的Query string # 详见https://www.elastic.co/guide/en/elasticsearch/reference/7.12/query-dsl-query-string-query.html#query-string-syntax elasticsearch { hosts => ["host:9200"] index => "python-log-auth-*" query => 'type:auth AND name:"%{[name]}"' fields => { "device" => "device" } sort => "@timestamp:desc" } # ruby ruby { code => "event.set('date', (event.get('message')[0..-7].to_f + event.get('message')[-6..-1].to_f / 1000000).to_s)" } } output { # debug output stdout { codec => rubydebug } 监控 curl -XGET 'localhost:9600/_node/stats/<ps|jvm|process|events|pipelines|reloads>' References Logstash 最佳实践