elasticsearch howeaver

------------------------------- elasticsearch

download zip

elasticsearch.yml:

# allow CORS requests from https://tauri.localhost

http.cors.enabled: true

http.cors.allow-origin: "https://tauri.localhost"


# and if your cluster uses authorization:

http.cors.allow-headers: X-Requested-With,Content-Type,Content-Length,Authorization


action.auto_create_index: .monitoring*,.watches,.triggered_watches,.watcher-history*,.ml*



roles.yml:

# The default roles file is empty as the preferred method of defining roles is

# through the API/UI. File based roles are useful in error scenarios when the

# API based roles may not be available.

admins:

  cluster:

    - all

  indices:

    - names:

        - "*"

      privileges:

        - all

devs:

  cluster:

    - manage

  indices:

    - names:

        - "*"

      privileges:

        - write

        - delete

        - create_index

-------------------------------

-------------------------------


-------------------------------

kibana 

zip

run

-------------------------------

-------------------------------


-------------------------------logstash

zip


csv:

145 deu Ich sag's ja nur! MUIRIEL 2010-10-14 14:43:54 2010-10-14 14:43:54

146 deu Wahrscheinlich war es das, was ihre Entscheidung beeinflusst hat. MUIRIEL 2010-10-14 14:43:54 2010-10-14 14:44:06

147 deu Ich habe mich immer gefragt, wie es wäre, Geschwister zu haben. MUIRIEL 2010-10-14 14:43:54 2010-10-14 14:44:19


config/import_csv.conf

input {

    file {

        path => "C:/app/logstash-8.5.3/sentences_detailed.csv" !!!!!!!!! on windows exactly forward slashes!!!!!

        start_position => beginning

        

}

}

filter {

    csv {

columns => ["id","lang","text","user_name","ts_add","ts_update"]

separator => " " tab symbol - not \t but just click tab symbol on a keyboard

        }


mutate {convert => ["id", "integer"]}


}

output {

elasticsearch {

  hosts => ["https://localhost:9200"]

  index => "sentences_detailed"

  ilm_enabled => false

  ssl => true

  ssl_certificate_verification => false

  user => "elastic"

  password => "2N-MgyiQApBRI9Eh3Bfn"

}

    stdout {}

}

-------------------------------

-------------------------------


execution:

elasticsearch 

    reset password elasticsearch-reset-password.bat for user elastic get password

kibana

C:\app\logstash-8.5.3>.\bin\logstash -f .\config\import_csv.conf


reimport:

kibana open in unsecured browser chrome

"C:\Program Files\Google\Chrome\Application\chrome.exe" --user-data-dir=C:\Users\User\chromeTemp --disable-web-security

: http://localhost:5601/app/dev_tools#/console

delete index: DELETE sentences_detailed

GET sentences_detailed

C:\app\logstash-8.5.3\data\plugins\inputs\file !!!!delete file - this is the log of imported files!

import:

C:\app\logstash-8.5.3>.\bin\logstash -f .\config\import_csv.conf

src: https://www.dataneb.com/post/loading-csv-data-into-elasticsearch-with-logstash

https://localhost:9200/ elasticsearch check in browser



для logstash 7.*

input {

    file {

        path => "C:/Users/Ya/Downloads/sentences_detailed/sentences_detailed.csv"

        start_position => beginning

        

}

}

filter {

    csv {

columns => ["id","lang","text","user_name","ts_add","ts_update"]

separator => " "

        }


mutate {convert => ["id", "integer"]}


}

output {

elasticsearch {

  hosts => ["http://localhost:9200"]

  index => "tatoeba_sentences_detailed"

}

    stdout {}

}

Комментарии

Отправить комментарий

Популярные сообщения из этого блога

kafka конспект однако

------------------------ part 1 ---------------------  schema evolution 1: avro - в сообщении содержится схема данных. предоставляет API для сериализации данных соответственно схеме и десериализации по схеме. exactly once processing: тут проблема в том что нет контроля над тем какой partition будет назначен. есть две стратегии: range: round robin: rebalance listener: случай когда consumer group coordinator запустил rebalance и consumer, обработавший часть сообщений хочет commit offset. чтобы это сделать надо знать: how to commit a particular offset? how to know that a rebalance is triggered? ConsumerRebalanceListener onPartitionsRevoked - будет вызван перед переключением partition. onPartitionsAssigned - будет вызвано после назначения partition и перед передачей сообщений. offset management: current offset: kafka broker у себя хранит указатель на на отправленные сообщения клонкретному consumer. maintain offset from which send next banch of ...
Далее...

debian, ubuntu 13.10 xrdp

variant 1: http://scarygliders.net/x11rdp-o-matic-central-downloads-page/ to remove use: dpkg -r xrdp variant 2: install xrdp: sudo apt-get install xrdp customize port(3389) http://manpages.ubuntu.com/manpages/hardy/man5/xrdp.ini.5.html#contenttoc0 /etc/xrdp/xrdp.ini install desctop environment: if need customize keyboard layout go http://c-nergy.be/blog/?p=4448 sudo add-apt-repository ppa:xubuntu-dev/xfce-4.10 sudo apt-get update sudo apt-get install xfce4 echo xfce4-session >~/.xsession sudo service xrdp restart
Далее...

sql optimizations

Изображение
B-tree is a self-balancing tree that maintains sorted data and allows operation in logarithmic time. B-trees can handle range queries on sorted data (<, ≤, >, ≥, between, in, is null, is not null) B-tree index can also be used for queries that involve pattern matching operator LIKE or ~ if the pattern is a constant and the anchor is at the beginning of the pattern. For example, you can try matching queries column_name LIKE ‘Apple%’ or column_name ~ ‘^Apple’ But, querying ‘%macbook%’ or ‘%pro’ will not be efficient. For such queries, the query planner will resort to full-text sequential search which is not optimized. Enter, GIN indexes. GIN stands for Generalized Inverted Indexes. We can create a GIN index to speed up text searches: CREATE INDEX index_name ON table USING GIN (to_tsvector(‘english’, column_name)); The query above specifies that the English configuration can be used to parse and normalize the strings. And for the part of searching, a simple query to print the tit...
Далее...