Removing Stop Words

I have a task to do word counting for some articles. The detail of the task is getting the list of id from Elasticsearch, get the content from ArangoDB, then do some text processing to clean the content and counting the word frequency. After did it with Scala, Go, and Python, I found out that it is very slow when I am doing it with Python. Doing it with Scala and Go only take around 3-4 seconds to process 12,563 articles. But when we do with Python, it takes around 15-18 seconds. And after do some profiling, finally I found out that it is very slow to remove any stopwords from big number of articles. I am using common method in Python to remove the stopwords. ...

October 20, 2018

Tensorflow GPU Memory Usage (Using Keras)

I am using Keras with Tensorflow backend for my project. In the beginning, when using Tensorflow backend, I am a bit surprised when seeing the memory usage. Although my model size is not more than 10 MB, It is still using all of my GPU memory. After reading tensorflow documentation, I found out, that by default, TensorFlow maps nearly all of the GPU memory. The purpose is to reduce the memory fragmentation. And also, from the documentation, I know there are two different approaches that can be used to handle this situation. ...

July 13, 2018

Gin Memcached Middleware

If you are using gin to build a webservice, and you want to use memcached to store your data, you will search some articles about how to create a middleware to do that. After read some articles, I only found this middleware. But I got no luck. I cannot get a value from a key on one endpoint after I set it on another endpoint. So I didn’t use gin session and create my own middleware with gomemcache. ...

December 6, 2017

Gogstash - Logstash Alternative

I am now doing some projects that need a monitoring application to monitor the webservice. After having some chit and chat, we decide to use ELK (Elasticsearch, Logstash, and Kibana). If you want to know what ELK is, just search on Google and there will be so many articles related to it. If you have already read some articles about ELK, you will know that ELK is the application to monitor and analyze all types of log. ...

December 5, 2017