SutoriaL

Tensorflow GPU Memory Usage (Using Keras)

No Comments

I am using Keras with Tensorflow backend for my project. In the beginning, when using Tensorflow backend, I am a bit surprised when seeing the memory usage. Although my model size is not more than 10 MB, It is still using all of my GPU memory.

After reading tensorflow documentation, I found out, that by default, TensorFlow maps nearly all of the GPU memory. The purpose is to reduce the memory fragmentation. And also, from the documentation, I know there are two different approaches that can be used to handle this situation.

The first method is limiting the memory usage by percentage. So, for example, you can limit the application just only use 20% of your GPU memory. If you are using 8GB GPU memory, the application will be using 1.4 GB. (I am using Keras, so the example will be done in Keras way)

import tensorflow as tf
from keras.backend.tensorflow_backend import set_session

config = tf.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction = 0.2
set_session(tf.Session(config=config))

After setting the configuration, we could see the difference.

It only uses 1.6 GB of memory. But, for this configuration, we only limit by percentage, and the application still use 100% of the limitation. We cannot know how much exactly the memory usage of the application. So we will try the second method.

The second method is using allow_growth. This method will make the application allocate only as much GPU memory based on runtime allocation. So, the application will be using the GPU memory as needed.

import tensorflow as tf
from keras.backend.tensorflow_backend import set_session

config = tf.ConfigProto()
config.gpu_options.allow_growth = True
sess = tf.Session(config=config)
set_session(sess)

And this is the result.

So, after trying two of these methods, which method will you use for your application? 😀

Continue Reading

Uncategorized

Syntaxnet with GPU Support

No Comments

After compiling Syntaxnet (old version with bazel 0.5.x) with GPU support, you will find this error message.

E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate xxxG (xxxxxxx bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate xxxG (xxxxxxx bytes) from device: CUDA_ERROR_OUT_OF_MEMORY

No, no. It is not because your GPU memory is not enough. It is because sometimes tensorflow eats all of your GPU memory. So, what do you have to do? You just have to modify the main function of this file.

models/research/syntaxnet/syntaxnet/parser_eval.py

gpu_opt = tf.GPUOptions(allow_growth=True)
with tf.Session(config=tf.ConfigProto(gpu_options=gpu_opt)) as sess:
    Eval(sess)

Or you can download the patch here.

Now you only use about 300 MB of your GPU memory to run Syntaxnet 😀

Reference: Github

Continue Reading

Uncategorized

Toml Config Value by Name (Go)

No Comments

I use https://github.com/BurntSushi/toml for parsing the configuration file. Because the library use the struct type for the configuration variable, we have to access the struct fields with a dot to get a value.

config.Database.Username

But I want to do it dynamically. I want to create a function that receive a string type key and return the configuration value. So I do it with this way.

package main

import (
	"fmt"
	"os"
	"reflect"
	"strings"
	"github.com/BurntSushi/toml"
)

type tomlConfig struct {
	Database databaseInfo
	Title    string
}

type databaseInfo struct {
	Username string
	Password string
}

// Function to get struct type variable by index name
func GetField(t *tomlConfig, field string) string {
	r := reflect.ValueOf(t)
	f := reflect.Indirect(r)

	splitField := strings.Split(field, ".")
	for _, s := range splitField {
		f = f.FieldByName(strings.Title(s))
	}

	return f.String()
}

// Function to get config by string
func GetConfig(key string) string {
	var config tomlConfig
	if _, err := toml.Decode(
		`
		title = "Text title"

		[database]
		username = "root"
		password = "password"
		`, &config); err != nil {
		fmt.Println("Please check your configuration file")
		os.Exit(1)
	}

	configValue := GetField(&config, key)

	return configValue
}

func main() {
	fmt.Println(GetConfig("testKey"))
	fmt.Println(GetConfig("database.testKey"))
	fmt.Println(GetConfig("title"))
	fmt.Println(GetConfig("database.username"))
	fmt.Println(GetConfig("database.password"))
}

Result:

$ go run main.go 
<invalid Value>
<invalid Value>
Text title
root
password
Continue Reading

SutoriaL

Gin Memcached Middleware

No Comments

If you are using gin to build a webservice, and you want to use memcached to store your data, you will search some articles about how to create a middleware to do that. After read some articles, I only found this middleware. But I got no luck. I cannot get a value from a key on one endpoint after I set it on another endpoint. So I didn’t use gin session and create my own middleware with gomemcache.

package main

import (
    "github.com/gin-gonic/gin"
    "github.com/bradfitz/gomemcache/memcache"
)

func MCMiddleware(mc *memcache.Client) gin.HandlerFunc {
    return func(c *gin.Context) {
        c.Set("mem", mc)
        c.Next()
    }
}

func main() {
    r := gin.Default()

    mc := memcache.New("127.0.0.1:11211")

    r.Use(MCMiddleware(mc))

    r.GET("/set", func(c *gin.Context) {
        mem, _ := c.MustGet("mem").(*memcache.Client)
        mem.Set(&amp;memcache.Item{Key: "somekey", Value: []byte("somevalue")})
        c.String(200, "ok\n")
    })

    r.GET("/get", func(c *gin.Context) {
        mem, _ := c.MustGet("mem").(*memcache.Client)
        data, _ := mem.Get("somekey")
        c.String(200, string([]byte(data.Value))+"\n")
    })

    r.Run(":8000")
}

Then you can set and get a value from a key on all endpoints.

linx@crawler ~ $ curl localhost:8000/set
ok
linx@crawler ~ $ curl localhost:8000/get
somevalue
Continue Reading