Friday, December 18, 2015

如何在caffe中增加layer以及caffe中triplet loss layer的实现

1.如何在caffe中增加新的layer

http://blog.csdn.net/tangwei2014/article/details/46812153

新版的caffe中增加新的layer，变得轻松多了，概括说来，分四步：

1）在./src/caffe/proto/caffe.proto 中增加对应layer的paramter message；

2）在./include/caffe/***layers.hpp中增加该layer的类的声明，***表示有common_layers.hpp, data_layers.hpp, neuron_layers.hpp, vision_layers.hpp 和loss_layers.hpp等；

3）在./src/caffe/layers/目录下新建.cpp和.cu文件，进行类实现。

4）在./src/caffe/gtest/中增加layer的测试代码，对所写的layer前传和反传进行测试，测试还包括速度。

最后一步很多人省了，或者没意识到，但是为保证代码正确，建议还是严格进行测试，磨刀不误砍柴功。

2.caffe中实现triplet loss layer

1.caffe.proto中增加triplet loss layer的定义

首先在message LayerParameter中追加 optional TripletLossParameter triplet_loss_param = 138; 其中138是我目前LayerParameter message中现有元素的个数，具体是多少，可以看LayerParameter message上面注释中的：

Tuesday, December 15, 2015

caffe HDF5 data layer preperation

https://groups.google.com/forum/#!topic/caffe-users/HN1eaUPBKO4

https://github.com/BVLC/caffe/tree/master/matlab/hdf5creation

Tuesday, December 1, 2015

Developing new layers

https://github.com/BVLC/caffe/wiki/Development

Caffe define a new layer hands on

Here's roughly the process I follow.

Add a class declaration for your layer to the appropriate one of common_layers.hpp,data_layers.hpp, loss_layers.hpp, neuron_layers.hpp, orvision_layers.hpp. Include an inline implementation of type and the *Blobs()methods to specify blob number requirements. Omit the *_gpu declarations if you'll only be implementing CPU code.
Implement your layer in layers/your_layer.cpp.
- SetUp for initialization: reading parameters, allocating buffers, etc.
- Forward_cpu for the function your layer computes
- Backward_cpu for its gradient
(Optional) Implement the GPU versions Forward_gpu and Backward_gpu inlayers/your_layer.cu.
Add your layer to proto/caffe.proto, updating the next available ID. Also declare parameters, if needed, in this file.
Make your layer createable by adding it to layer_factory.cpp.
Write tests in test/test_your_layer.cpp. Usetest/test_gradient_check_util.hpp to check that your Forward and Backward implementations are in numerical agreement.

Friday, September 4, 2015

useful dataset for cv

http://rogerioferis.com/VisualRecognitionAndSearch2014/Resources.html

how to debug matlab code in linxu system

x = rand(10,10);
y = rand(10,5);
z = x * y;
keyboard; % you can interactively inspect x, y, z here
z = sort(z);

To leave keyboard mode, you can type dbquit to exit the program, or return to continue executing the program.

Thursday, September 3, 2015

feature reduction code

http://www.cad.zju.edu.cn/home/dengcai/Data/DimensionReduction.html

Benchmarks and challenges like PASCAL VOC and ImageNet have played a crucial role in advancing computer vision algorithms. However, with minor exceptions, such challenges also result in massive duplication of effort, with each research group developing its own infrastructure and code-base. In fact, warnings of fragmentation and lack of code reuse have repeatedly been among the top observations by forward-looking NSF funded workshops [1,2].

CloudCV can help unify fragmented efforts by being a unified data and code repository.

Thursday, August 27, 2015

extract features using caffe

http://www.cnblogs.com/platero/p/3967208.html

feat_helper_pb2.py

# Generated by the protocol buffer compiler. DO NOT EDIT!

from google.protobuf import descriptor
from google.protobuf import message
from google.protobuf import reflection
from google.protobuf import descriptor_pb2
# @@protoc_insertion_point(imports)

DESCRIPTOR = descriptor.FileDescriptor(
name='datum.proto',
package='feat_extract',
serialized_pb='\n\x0b\x64\x61tum.proto\x12\x0c\x66\x65\x61t_extract\"i\n\x05\x44\x61tum\x12\x10\n\x08\x63hannels\x18\x01 \x01(\x05\x12\x0e\n\x06height\x18\x02 \x01(\x05\x12\r\n\x05width\x18\x03 \x01(\x05\x12\x0c\n\x04\x64\x61ta\x18\x04 \x01(\x0c\x12\r\n\x05label\x18\x05 \x01(\x05\x12\x12\n\nfloat_data\x18\x06 \x03(\x02')

_DATUM = descriptor.Descriptor(
name='Datum',
full_name='feat_extract.Datum',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
descriptor.FieldDescriptor(
name='channels', full_name='feat_extract.Datum.channels', index=0,
number=1, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
options=None),
descriptor.FieldDescriptor(
name='height', full_name='feat_extract.Datum.height', index=1,
number=2, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
options=None),
descriptor.FieldDescriptor(
name='width', full_name='feat_extract.Datum.width', index=2,
number=3, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
options=None),
descriptor.FieldDescriptor(
name='data', full_name='feat_extract.Datum.data', index=3,
number=4, type=12, cpp_type=9, label=1,
has_default_value=False, default_value="",
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
options=None),
descriptor.FieldDescriptor(
name='label', full_name='feat_extract.Datum.label', index=4,
number=5, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
options=None),
descriptor.FieldDescriptor(
name='float_data', full_name='feat_extract.Datum.float_data', index=5,
number=6, type=2, cpp_type=6, label=3,
has_default_value=False, default_value=[],
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
options=None),
],
extensions=[
],
nested_types=[],
enum_types=[
],
options=None,
is_extendable=False,
extension_ranges=[],
serialized_start=29,
serialized_end=134,
)

DESCRIPTOR.message_types_by_name['Datum'] = _DATUM

class Datum(message.Message):
__metaclass__ = reflection.GeneratedProtocolMessageType
DESCRIPTOR = _DATUM

# @@protoc_insertion_point(class_scope:feat_extract.Datum)

# @@protoc_insertion_point(module_scope)

leveldb2mat.py

import leveldb
import feat_helper_pb2
import numpy as np
import scipy.io as sio
import time

def main(argv):
leveldb_name = sys.argv[1]
print "%s" % sys.argv[1]
batch_num = int(sys.argv[2]);
batch_size = int(sys.argv[3]);
window_num = batch_num*batch_size;

start = time.time()
if 'db' not in locals().keys():
db = leveldb.LevelDB(leveldb_name)
datum = feat_helper_pb2.Datum()

ft = np.zeros((window_num, int(sys.argv[4])))
for im_idx in range(window_num):
datum.ParseFromString(db.Get('%d' %(im_idx)))
ft[im_idx, :] = datum.float_data

print 'time 1: %f' %(time.time() - start)
sio.savemat(sys.argv[5], {'feats':ft})
print 'time 2: %f' %(time.time() - start)
print 'done!'

#leveldb.DestroyDB(leveldb_name)

if __name__ == '__main__':
import sys

main(sys.argv)

#!/usr/bin/env sh
# args for EXTRACT_FEATURE
TOOL=../../build/tools
MODEL=../../examples/imagenet/caffe_reference_imagenet_model #下载得到的caffe model
PROTOTXT=../../examples/_temp/imagenet_val.prototxt # 网络定义
LAYER=conv1 # 提取层的名字，如提取fc7等
LEVELDB=../../examples/_temp/features_conv1 # 保存的leveldb路径
BATCHSIZE=10

# args for LEVELDB to MAT
DIM=290400 # 需要手工计算feature长度
OUT=../../examples/_temp/features.mat #.mat文件保存路径
BATCHNUM=1 # 有多少哥batch， 本例只有两张图， 所以只有一个batch

$TOOL/extract_features.bin  $MODEL $PROTOTXT $LAYER $LEVELDB $BATCHSIZE
python leveldb2mat.py $LEVELDB $BATCHNUM  $BATCHSIZE $DIM $OUT

caffe extract features - tool batch size and model batch size

https://github.com/BVLC/caffe/issues/585#issuecomment-51856173
Caffe operates on batches of data in GPU mode to reduce the overhead of communicating between the host and device. The feature extraction example batch_size is set to 50, and when the tool is called it is given 10 iterations as its argument, so it is actually processing 500 inputs. Set the batch_size and iteration arg to both be 1 to process a single image, although that's less efficient in throughput.

batch_size is set to 50
the tool batch size is set to 10 then it will extract 500 image features
the tool batch size is set to 20 then it will extract 1000 image features

juana