Kibana: Create index patterns and set default index pattern through curl

I needed programatically create kibana index patterns and set the default index pattern to support an automated deployment.  The below two curl commands were used in a bash script to do that.  To get the $KIBANA_HIDDEN_INDEX, navigate to <kibana_home>/config/kibana.yml, and note down the "kibana.index" value.

Environment, this worked on both:
centos-release-6-8.el6.centos.12.3.x86_64
kibana-5.0.2-linux-x86_64

Windows 10 with Git Bash (Cygwin)
kibana-5.0.2-windows-x86

1. Create kibana index pattern:

curl -X POST -H "kbn-xsrf:true" -H "Content-Type: application/json" -d '{"timeFieldName":"@timestamp","title": "$INDEX_PATTERN"}' http://localhost:5601/elasticsearch/$KIBANA_HIDDEN_INDEX/index-pattern/$INDEX_PATTERN?op_type=create

curl -X POST -H "kbn-xsrf:true" -H "Content-Type: application/json" -d '{"timeFieldName":"@timestamp","title": "cherryshoe_idx"}' http://localhost:5601/elasticsearch/.local/index-pattern/cherryshoe_idx?op_type=create


2. Set default index pattern:
curl -X POST -H "Content-Type: application/json" -H "kbn-xsrf: true" -d '{"value":"$INDEX_PATTERN"}' http://localhost:5601/api/kibana/settings/defaultIndex

curl -X POST -H "Content-Type: application/json" -H "kbn-xsrf: true" -d '{"value":"cherryshoe_idx"}' http://localhost:5601/api/kibana/settings/defaultIndex

Updated instructions for Kibana 6.4.2:
Windows 10 with Git Bash (Cygwin)
kibana-6.4.2-windows-x86_64

1. Create kibana index pattern:

jsonValue="{\"attributes\":{\"title\":\"$INDEX_PATTERN\",\"timeFieldName\":\"@timestamp\"}}"
curl -X POST -H "kbn-xsrf:true" -H "Content-Type: application/json" -d $jsonValue http://localhost:5601/api/saved_objects/index-pattern/$INDEX_PATTERN

jsonValue="{\"attributes\":{\"title\":\"cherryshoe_idx\",\"timeFieldName\":\"@timestamp\"}}"
curl -X POST -H "kbn-xsrf:true" -H "Content-Type: application/json" -d $jsonValue http://localhost:5601/api/saved_objects/index-pattern/cherryshoe_idx


2. Set default index pattern:
defaultIndexValue="{\"changes\":{\"defaultIndex\":\"$INDEX_PATTERN\"}}"
curl -X POST -H "kbn-xsrf:true" -H "Content-Type: application/json" -d $defaultIndexValue http://localhost:5601/api/kibana/settings

defaultIndexValue="{\"changes\":{\"defaultIndex\":\"cherryshoe_idx\"}}"
curl -X POST -H "kbn-xsrf:true" -H "Content-Type: application/json" -d $defaultIndexValue http://localhost:5601/api/kibana/settings

This article helped a lot.

Bash script to call rest endpoint for each id in text file

I needed to write an admin utility script to delete documents from a content management system.  First, the list of document ids were retrieved from the application database; Second, the list of ids were used as input to a bash script to make REST calls to delete the documents.

Environment:
MySql 5.7.10
RHEL 7

STEPS:

1. Retrieve list of ids from the database and save to uuids.txt, one id on each line. The ids were alfresco cmis unique ids (with unique nodeRef id, ;,  and version number), they look like "workspace://SpacesStore/aa3e765f-c628-44c2-bc66-17d622ca2210;1.0".  I needed documents created after a certain date, and only the unique id portion after the "SpacesStore/" and prior to the ";1.0" of the unique id.  Below is the MySQL select statement:
   
   select substring_index(substring_index(alf_node_id, 'SpacesStore/', -1),
                          ';1.0', 1), created_dt from cherryshoe_documents where created_dt >= '2017-03-18 00:00:00' order by created_dt desc;
   
   
2. The bash script is below, it incorporates a curl call to a REST DELETE endpoint with basic authentication.
   
   delete.sh

   
   #!/usr/bin/bash
   # Calls a repository rest endpoint to delete node(s).
   # Takes in an input text file of one id per line.
   
   username=admin
   password=admin
   protocol=http
   #hostname can contain port
   hostname="localhost:8080"
   
   echo username=$username, password=$password, protocol=$protocol, hostname=$hostname
   
   if (( "$#" != 1 ))
   then
       echo "Usage Info: Enter in filename"
   exit 1
   fi
   
   filename="$1"
   
   echo "Starting Node Delete..."
   while read -r line
   do
       uuid="$line"
       endpoint=$protocol://$hostname/alfresco/s/api/node/workspace/SpacesStore/$uuid
       # contains a \r (CR) at the end (0d). Remove it with
       endpoint=${endpoint%$'\r'}
       echo $endpoint
   
       echo "#######START $uuid"
       curl -u $username:$password -i -X DELETE $endpoint
       echo "#######END $uuid"
   
   done < "$filename"
   
   echo "DONE"
   

   
   
3. Sample uuids.txt file below:
   32231a66-649a-4cee-9a16-2c45b639fd94
   08b53382-e120-4100-a093-32b2a57b9bad
   6759904f-48bf-4e02-93e0-6ee9f9885df7
   9931f0e8-1b48-4c3c-a885-94ccf80866ea
   aedef20e-6831-4fdc-ad21-872af333d985
   
   
   

Cross-domain AJAX calls with JSONP

It's a common issue to have a web application running on a server that needs to make a web service call to a service running on another server. There are a couple of solutions to that:

1. Make a direct HTTP GET call with server side code to the service:  
• Pro: No special processing of the JSON return data: make the HTTP GET call, process the JSON response, display the data
• Con: This is limited, as the page being returned is static
2. Make an AJAX call with client side code to the service:  
• Pro: Dynamically make the service call based on user interaction
• Con: The application needs to have specialized handling to make the cross-domain AJAX request

For this discussion, we'll look deeper at the second bullet and explore the solution further.

Cross-Domain AJAX calls with JSONP
Without properly setting things up to make cross-domain AJAX calls, you might see the following if using Firebug.  You might see the following and wonder what the heck is going on?

• Firebug shows a HTTP OK 200 response, which makes it appear that the call was successful.  But on closer inspection, there's no data returned in the response...
• Firebug shows that data was returned, but it says the JSON is invalid.  When checking the JSON data in jsonlint, it's perfectly valid JSON!

JSONP (JSON with padding) can help us implement this more appropriately, but there are a couple of caveats:

Caveat 1: You have to be in control of the web application that is making the AJAX request:

• That is because you need to specify to the web service that you want to wrap the response as JSONP.  
• If using jQuery (since version 1.2), you can load JSON data located on another domain if you specify a JSONP "callback" URL parameter.

/webapp/service/com/blogspot/cherryshoe/docList?id=06022012&callback=?

Caveat 2: You have to be in control of the web service that is being called:

• That is because you need to look for that optional "callback" parameter; if it's available, wrap the response as JSONP.
• If using jQuery, jQuery will automatically assign the "callback" parameter a value, in this case it was "jQuery16406082620163990349_1361055846009", the web service will wrap the JSON response with that value, making it JSONP.

jQuery16406082620163990349_1361055846009
(
    {
        "status" : true,
        "statusMessage" : "Retrieved 2 documents for id: 06022012",
        "totalRecords" : 2,
        "documents" : 
        [
        {
            "docId" : "1234",
            "docName" : "06022012_1.pdf",
            "docType" : "1"
        },
        {
            "docId" : "5678",
            "docName" : "06022012_2.pdf",
            "docType" : "2"
        }
        ]
    }
)

Obviously there may be security considerations if the web service or web application is not public.  This article helped a lot when I was researching this issue.