Syntaxnet with GPU Support

After compiling Syntaxnet (old version with bazel 0.5.x) with GPU support, you will find this error message.

E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate xxxG (xxxxxxx bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate xxxG (xxxxxxx bytes) from device: CUDA_ERROR_OUT_OF_MEMORY

No, no. It is not because your GPU memory is not enough. It is because sometimes tensorflow eats all of your GPU memory. So, what do you have to do? You just have to modify the main function of this file.

models/research/syntaxnet/syntaxnet/parser_eval.py

gpu_opt = tf.GPUOptions(allow_growth=True)
with tf.Session(config=tf.ConfigProto(gpu_options=gpu_opt)) as sess:
    Eval(sess)

Or you can download the patch here.

Now you only use about 300 MB of your GPU memory to run Syntaxnet 😀

Reference: Github

Continue Reading

Toml Config Value by Name (Go)

I use https://github.com/BurntSushi/toml for parsing the configuration file. Because the library use the struct type for the configuration variable, we have to access the struct fields with a dot to get a value.

config.Database.Username

But I want to do it dynamically. I want to create a function that receive a string type key and return the configuration value. So I do it with this way.

package main

import (
	"fmt"
	"os"
	"reflect"
	"strings"
	"github.com/BurntSushi/toml"
)

type tomlConfig struct {
	Database databaseInfo
	Title    string
}

type databaseInfo struct {
	Username string
	Password string
}

// Function to get struct type variable by index name
func GetField(t *tomlConfig, field string) string {
	r := reflect.ValueOf(t)
	f := reflect.Indirect(r)

	splitField := strings.Split(field, ".")
	for _, s := range splitField {
		f = f.FieldByName(strings.Title(s))
	}

	return f.String()
}

// Function to get config by string
func GetConfig(key string) string {
	var config tomlConfig
	if _, err := toml.Decode(
		`
		title = "Text title"

		[database]
		username = "root"
		password = "password"
		`, &config); err != nil {
		fmt.Println("Please check your configuration file")
		os.Exit(1)
	}

	configValue := GetField(&config, key)

	return configValue
}

func main() {
	fmt.Println(GetConfig("testKey"))
	fmt.Println(GetConfig("database.testKey"))
	fmt.Println(GetConfig("title"))
	fmt.Println(GetConfig("database.username"))
	fmt.Println(GetConfig("database.password"))
}

Result:

$ go run main.go 
<invalid Value>
<invalid Value>
Text title
root
password
Continue Reading

Gogstash – Logstash Alternative

I am now doing some projects that need a monitoring application to monitor the webservice. After having some chit and chat, we decide to use ELK (Elasticsearch, Logstash, and Kibana). If you want to know what ELK is, just search on Google and there will be so many articles related to it.

If you have already read some articles about ELK, you will know that ELK is the application to monitor and analyze all types of log.

  • Elasticsearch: indexing the data.
  • Logstash: log processing / parsing.
  • Kibana: visualize the data.

But after trying to configure and run ELK, I found out that Logstash is heavy to be run on the server with small specification. Because of this reason, I am trying to find some Logstash alternatives, and finally I found Gogstash, Logstash like, written in Golang.

While reading the documentation, I found out that there are some differences between Gogstash and Logstash when using the filter (I am using grok filter in Logstash). I tried to apply same pattern in Gogstash but it didn’t work. After all these things, I decide to use another filter. I am using gonx filter.

Although grok pattern and gonx pattern is different, it is not so difficult to create the configuration for gonx filter. And after some modification, Gogstash run smoothly. For your information, I am using flask for building the webservice, and this is an example line of the application log.

192.168.100.57 - - [05/Dec/2017 16:27:27] "GET / HTTP/1.1" 200 -

There are two types of Gogstash configuration, json and yml format. Here is my yml configuration.

input:
  - type: file
    path: '/home/linggar/webapp/nohup.out'

filter:
  - type: gonx
    format: '$clientip - - [$date $time] "$full_request" $status -'
    source: message
  - type: gonx
    format: '$verb $request HTTP/$httpversion'
    source: full_request

output:
  - type: elastic
    url: 'http://127.0.0.1:9200'
    index: gogstash_log
    document_type: testtype

And that’s it. Although Gogstash is not as powerful as Logstash, it is very light and one of so many Logstash alternatives which you could try.

Continue Reading

Ebean Partial Column Selection

Okay, because of some reasons, I have to learn Java and choose a Java web framework for my next project. After seeing around, I choose playframework, and I am using MySQL for the database.

After reading some documentation, I use Evolution and Ebean for the ORM. And then I found a problem when using Ebean. It still returning all columns although I selected some columns with this method based on this documentation.

List<MyModel> getUser = MyModel.find.select("columnA, columnB").findList()

After searching some solutions on google, I found this solution. So you have to do it like this.

JsonContext jc = Ebean.json();
List<MyModel> getUser = MyModel.find.select("columnA, columnB").findList();
String allUser = jc.toJson(getUser);

But when you use this method, the type of result will be a string type, so do not forget using Json.parse() if you want to use it as an object.

Continue Reading