NiFi Custom Processor 만들기

Custom Processor 을 만드는 이유
준비
프로젝트 생성
[샘플] 프로젝트 제작
프로젝트 빌드
jar:1.0-SNAPSHOT 오류 해결방법
프로젝트 빌드 - 성공
적용하기

Custom Processor 을 만드는 이유

NiFi 에는 340개 이상의 기본(기본 설치시 있는) Processor들이 있지만, 직접 내가 Processor을 만들 수도 있다.

Custom Processor을 만들 필요성은 아래와 같다.

ExecuteScript보다 더 디테일한 부분(관계, properties설정 등)기능이 필요할 때
ExecuteScript의 단점(코드노출 및 인터프리터 언어 등)을 보완하고 싶을때
Java로 만들고 싶을때
NiFi 기본 프로세서로 없는 프로세서를 만들고 배포하고 싶을 때
등

단순한 관계설정(실패, 성공), 매우 간단한 로직, 파이썬이나 자바스크립트를 사용하고 싶은 경우에는 ExecuteScript를 사용하는 것도 방법 중 하나이다.

아래 링크에 ExecuteScript를 사용하는 방법에 대해 설명한다.

https://hbcha0916.tistory.com/26

NiFi - ExecuteScript (NiFi에서 Python사용하기)

NiFi - ExecuteScript (NiFi에서 Python사용하기) NiFi에서 내장된 프로세서로는 데이터흐름 및 처리가 어려울 때 직접 코딩하여 FlowFile을 제어, 변형할 수 있는 프로세서이다. 만약에 더 높은 수준으로 Flow

hbcha0916.tistory.com

준비

우선 아래와 같은 준비가 되어있는지 확인한다.

Apache Maven이 설치되어 있는가?
내가 만들 프로세서가 이미 있지는 않은가?
자바 IDE(통합 개발환경 Eclipse, VScode.. 등)이 설치되어 있는가?

Apache Maven이 설치되어 있지 않으면 아래 링크에서 설치한다.

https://hbcha0916.tistory.com/34

[Apache Maven] Apache Maven 설치하기

NiFi에서 커스텀 프로세서를 만들기 위해서는 Apache Maven을 이용하여야 하는데 아직 내 컴퓨터에는 Maven이 설치되어 있지 않다. 공식 홈페이지 및 다운로드 아래 사이트에서 Binary zip archive 혹은 Bina

hbcha0916.tistory.com

프로젝트 생성

ApacheMaven으로 새로운 프로세서에 대한 프로젝트를 생성한다.

우선 윈도우든, 리눅스든 `cd`명령어로 새로운 프로젝트를 생성할 디렉토리로 이동한다.
난 (C:\Users\user) 디렉토리에다가 만들겠다.

이동 후 아래와 같은 명령어를 입력한다.

mvn archetype:generate

"Choose a number or apply filter"라는 입력이 나오면 `nifi` 입력(아래는 예)

Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): 2051: nifi

"Choose archetype"라는 입력이 나오면 "remote -> org.apache.nifi:nifi-processor-bundle-archetype"가 있는 번호를 선택하고 엔터(아래는 예)

Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): 2051: nifi
Choose archetype:
1: remote -> org.apache.nifi:nifi-processor-bundle-archetype (-)
2: remote -> org.apache.nifi:nifi-service-bundle-archetype (-)
Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): : 1

"Choose org.apache.nifi:nifi-processor-bundle-archetype version"라는 입력이 나오면 적용시킬 NiFi 버전 번호를 입력한다. (아래는 예) 나는 1.20.0 버전이라 "58"을 입력했다.

Choose org.apache.nifi:nifi-processor-bundle-archetype version:
1: 0.0.2-incubating
2: 0.1.0-incubating
3: 0.2.0-incubating
4: 0.2.1
5: 0.3.0
6: 0.4.0
7: 0.4.1
8: 0.5.0
9: 0.5.1
10: 0.6.0
11: 0.6.1
12: 0.7.0
13: 0.7.1
14: 0.7.2
15: 0.7.3
16: 0.7.4
17: 1.0.0-BETA
18: 1.0.0
19: 1.0.1
20: 1.1.0
21: 1.1.1
22: 1.1.2
23: 1.2.0
24: 1.3.0
25: 1.4.0
26: 1.5.0
27: 1.6.0
28: 1.7.0
29: 1.7.1
30: 1.8.0
31: 1.9.0
32: 1.9.1
33: 1.9.2
34: 1.10.0
35: 1.11.0
36: 1.11.1
37: 1.11.2
38: 1.11.3
39: 1.11.4
40: 1.12.0
41: 1.12.1
42: 1.13.0
43: 1.13.1
44: 1.13.2
45: 1.14.0
46: 1.15.0
47: 1.15.1
48: 1.15.2
49: 1.15.3
50: 1.16.0
51: 1.16.1
52: 1.16.2
53: 1.16.3
54: 1.17.0
55: 1.18.0
56: 1.19.0
57: 1.19.1
58: 1.20.0
59: 1.21.0
Choose a number: 59: 58

"Define 뭐시기~"라는 입력이 나오면 아래 예와 규칙을 이용해 적절히 생성해 준다.

Define value for property 'artifactBaseName': hbcha
[INFO] Using property: nifiVersion = 1.20.0
Define value for property 'groupId': hbcha0916.tistory.com
Define value for property 'artifactId': my-test-processor
Define value for property 'version' 1.0-SNAPSHOT: : #그냥 엔터했었다.
Define value for property 'package' hbcha0916.tistory.com.processors.hbcha: : myprocessor
Confirm properties configuration:
artifactBaseName: hbcha
nifiVersion: 1.20.0
groupId: hbcha0916.tistory.com
artifactId: my-test-processor
version: 1.0-SNAPSHOT
package: myprocessor

GroupID	새 번들의 Maven Group
ArtifactID	특수 문자를 사용하지 않고 소문자와 하이픈 으로만 작성
Version	숫자와 점으로 구성된 버전(예- 1.1, 1.3…)
ArtifactBaseName	번들을 설명, ArtifactID와 일치해야 한다.
Package	프로세서용 JAVA패키지

"Y"라는 입력이 나오면 똑같이 Y를 입력해 준다. (아래는 예)

Y: : Y

그럼 좌라락 하면서 Maven이 프로젝트를 생성시킨다.(아래는 예)

[INFO] ----------------------------------------------------------------------------
[INFO] Using following parameters for creating project from Archetype: nifi-processor-bundle-archetype:1.20.0
[INFO] ----------------------------------------------------------------------------
[INFO] Parameter: groupId, Value: hbcha0916.tistory.com
[INFO] Parameter: artifactId, Value: my-test-processor
[INFO] Parameter: version, Value: 1.0-SNAPSHOT
[INFO] Parameter: package, Value: myprocessor
[INFO] Parameter: packageInPathFormat, Value: myprocessor
[INFO] Parameter: package, Value: myprocessor
[INFO] Parameter: artifactBaseName, Value: hbcha
[INFO] Parameter: groupId, Value: hbcha0916.tistory.com
[INFO] Parameter: artifactId, Value: my-test-processor
[INFO] Parameter: nifiVersion, Value: 1.20.0
[INFO] Parameter: version, Value: 1.0-SNAPSHOT
[INFO] Project created from Archetype in dir: C:\Users\user\my-test-processor
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  11:29 min
[INFO] Finished at: 2023-05-31T13:49:15+09:00
[INFO] ------------------------------------------------------------------------
[WARNING]
[WARNING] Plugin validation issues were detected in 2 plugin(s)
[WARNING]
[WARNING]  * org.apache.maven.plugins:maven-site-plugin:3.12.1
[WARNING]  * org.apache.maven.plugins:maven-archetype-plugin:3.2.1
[WARNING]
[WARNING] For more or less details, use 'maven.plugin.validation' property with one of the values (case insensitive): [BRIEF, DEFAULT, VERBOSE]
[WARNING]

[WARNING] 이 나오는데 해결하고 싶었지만 일단 프로젝트를 만드는데 문제가 없었으므로 넘어간다.

ArtifactID에 입력했던 문자로 해당 작업 디렉토리에 프로젝트가 생성된 것을 확인할 수 있었다. (아래는 powershell에서 확인한 결과)

PS C:\Users\user> ls | select-string -pattern my-test

my-test-processor

[샘플] 프로젝트 제작

아주 간단한 샘플 프로젝트를 제작해 보자

단순히 properties와 relationship을 추가해 보자

이클립스 혹은 VScode 등 IDE에서 해당 프로젝트를 열기

${ArtifactID} > nifi - ${ArtifactBaseName} - nar > src > main > java > ${Package} 디렉터리 내에 MyProcessor.java 가 있다.

(위 예로 예를 들면 "C:\Users\user\my-test-processor\nifi-hbcha-processors\src\main\java\myprocessor" 디렉토리이다.)

해당 파일을 열면 아래와 같이 나와있다.

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package myprocessor;

import org.apache.nifi.components.PropertyDescriptor;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.annotation.behavior.ReadsAttribute;
import org.apache.nifi.annotation.behavior.ReadsAttributes;
import org.apache.nifi.annotation.behavior.WritesAttribute;
import org.apache.nifi.annotation.behavior.WritesAttributes;
import org.apache.nifi.annotation.lifecycle.OnScheduled;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.SeeAlso;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.processor.exception.ProcessException;
import org.apache.nifi.processor.AbstractProcessor;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.ProcessSession;
import org.apache.nifi.processor.ProcessorInitializationContext;
import org.apache.nifi.processor.Relationship;
import org.apache.nifi.processor.util.StandardValidators;

import java.util.ArrayList;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Set;

@Tags({"example"})
@CapabilityDescription("Provide a description")
@SeeAlso({})
@ReadsAttributes({@ReadsAttribute(attribute="", description="")})
@WritesAttributes({@WritesAttribute(attribute="", description="")})
public class MyProcessor extends AbstractProcessor {

    public static final PropertyDescriptor MY_PROPERTY = new PropertyDescriptor
            .Builder().name("MY_PROPERTY")
            .displayName("My property")
            .description("Example Property")
            .required(true)
            .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
            .build();

    public static final Relationship MY_RELATIONSHIP = new Relationship.Builder()
            .name("MY_RELATIONSHIP")
            .description("Example relationship")
            .build();

    private List<PropertyDescriptor> descriptors;

    private Set<Relationship> relationships;

    @Override
    protected void init(final ProcessorInitializationContext context) {
        descriptors = new ArrayList<>();
        descriptors.add(MY_PROPERTY);
        descriptors = Collections.unmodifiableList(descriptors);

        relationships = new HashSet<>();
        relationships.add(MY_RELATIONSHIP);
        relationships = Collections.unmodifiableSet(relationships);
    }

    @Override
    public Set<Relationship> getRelationships() {
        return this.relationships;
    }

    @Override
    public final List<PropertyDescriptor> getSupportedPropertyDescriptors() {
        return descriptors;
    }

    @OnScheduled
    public void onScheduled(final ProcessContext context) {

    }

    @Override
    public void onTrigger(final ProcessContext context, final ProcessSession session) {
        FlowFile flowFile = session.get();
        if ( flowFile == null ) {
            return;
        }
        // TODO implement
    }
}

ExecuteScript를 사용해 본 사람은 대충 어떤지 보일 것이다.

NiFi 프로세서는 `onTrigger` 에서 실행된다.

위 코드에서 아래 코드로 변경시켜 본다.(전부 복붙 할 경우에는 `package` 부분만 빼고 붙여놓는다.)

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package myprocessor;

import org.apache.nifi.components.PropertyDescriptor;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.annotation.behavior.ReadsAttribute;
import org.apache.nifi.annotation.behavior.ReadsAttributes;
import org.apache.nifi.annotation.behavior.WritesAttribute;
import org.apache.nifi.annotation.behavior.WritesAttributes;
import org.apache.nifi.annotation.lifecycle.OnScheduled;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.SeeAlso;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.processor.exception.ProcessException;
import org.apache.nifi.processor.AbstractProcessor;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.ProcessSession;
import org.apache.nifi.processor.ProcessorInitializationContext;
import org.apache.nifi.processor.Relationship;
import org.apache.nifi.processor.util.StandardValidators;

import java.util.ArrayList;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Set;

@Tags({"example"})
@CapabilityDescription("Provide a description")
@SeeAlso({})
@ReadsAttributes({@ReadsAttribute(attribute="", description="")})
@WritesAttributes({@WritesAttribute(attribute="", description="")})
public class MyProcessor extends AbstractProcessor {

    public static final PropertyDescriptor PROPERTY1 = new PropertyDescriptor
            .Builder().name("MY_PROPERTY1")
            .displayName("My property1")
            .description("Example Property1")
            .required(true)
            .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
            .build();

    public static final PropertyDescriptor PROPERTY2 = new PropertyDescriptor
            .Builder().name("MY_PROPERTY2")
            .displayName("My property2")
            .description("Example Property2")
            .required(true)
            .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
            .build();

    public static final Relationship RELATIONSHIP1 = new Relationship.Builder()
            .name("MY_RELATIONSHIP1")
            .description("Example relationship1")
            .build();

    public static final Relationship RELATIONSHIP2 = new Relationship.Builder()
            .name("MY_RELATIONSHIP2")
            .description("Example relationship2")
            .build();

    private List<PropertyDescriptor> descriptors;

    private Set<Relationship> relationships;

    @Override
    protected void init(final ProcessorInitializationContext context) {
        descriptors = new ArrayList<>();
        descriptors.add(PROPERTY1);
        descriptors.add(PROPERTY2);
        descriptors = Collections.unmodifiableList(descriptors);

        relationships = new HashSet<>();
        relationships.add(RELATIONSHIP1);
        relationships.add(RELATIONSHIP2);
        relationships = Collections.unmodifiableSet(relationships);
    }

    @Override
    public Set<Relationship> getRelationships() {
        return this.relationships;
    }

    @Override
    public final List<PropertyDescriptor> getSupportedPropertyDescriptors() {
        return descriptors;
    }

    @OnScheduled
    public void onScheduled(final ProcessContext context) {

    }

    @Override
    public void onTrigger(final ProcessContext context, final ProcessSession session) {
        FlowFile flowFile = session.get();
        if ( flowFile == null ) {
            return;
        }
        // TODO implement
    }
}

Relationship 함수, PropertyDescriptor 함수, init 함수에서 descriptors값이 추가된 것을 알 수 있다.

이렇게 만들고 저장하자.

참고
FlowFile을 제어하는 구문은 `onTrigger` 함수에서 `// TODO implement` 부분을 지우고 코딩하면 된다.

프로젝트 빌드

터미널에서 프로젝트의 최상위 디렉토리 ${ArtifactID}로 이동한다.
나 같은 경우는 (C:\Users\user\my-test-processor) 디렉터리이다.

아래 명령어를 입력하여 프로젝트를 빌드한다.

mvn clean install

아래와 같이 좌라락 빌드를 시작한다.

PS C:\Users\user\my-custom-processor> cd ..\my-test-processor\
PS C:\Users\user\my-test-processor> mvn clean install
[INFO] Scanning for projects...
[INFO] Inspecting build with total of 3 modules...
[INFO] Installing Nexus Staging features:
[INFO]   ... total of 3 executions of maven-deploy-plugin replaced with nexus-staging-maven-plugin
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] my-test-processor                                                  [pom]
[INFO] nifi-hbcha-processors                                              [jar]
[INFO] nifi-hbcha-nar                                                     [nar]
[INFO]
[INFO] --------------< hbcha0916.tistory.com:my-test-processor >---------------
[INFO] Building my-test-processor 1.0-SNAPSHOT                            [1/3]
[INFO]   from pom.xml
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- clean:3.2.0:clean (default-clean) @ my-test-processor ---
[INFO]
[INFO] --- enforcer:3.2.1:enforce (enforce-maven-version) @ my-test-processor ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.BanDuplicatePomDependencyVersions passed
[INFO] Rule 1: org.apache.maven.enforcer.rules.RequireSameVersions passed
[INFO] Rule 2: org.apache.maven.enforcer.rules.version.RequireMavenVersion passed
[INFO] Rule 3: org.apache.maven.enforcer.rules.dependency.RequireReleaseDeps passed
[INFO] Rule 4: org.apache.maven.enforcer.rules.dependency.BannedDependencies passed
[INFO]
[INFO] --- enforcer:3.2.1:enforce (enforce-java-version) @ my-test-processor ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.version.RequireJavaVersion passed
[INFO]
[INFO] --- buildnumber:3.0.0:create (default) @ my-test-processor ---
[INFO]
[INFO] --- remote-resources:1.7.0:process (process-resource-bundles) @ my-test-processor ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- compiler:3.10.1:testCompile (groovy-tests) @ my-test-processor ---
[INFO] No sources to compile
[INFO]
[INFO] --- site:3.12.1:attach-descriptor (attach-descriptor) @ my-test-processor ---
[INFO] No site descriptor found: nothing to attach.
[INFO]
[INFO] --- install:3.1.0:install (default-install) @ my-test-processor ---
[INFO] Installing C:\Users\user\my-test-processor\pom.xml to C:\Users\user\.m2\repository\hbcha0916\tistory\com\my-test-processor\1.0-SNAPSHOT\my-test-processor-1.0-SNAPSHOT.pom
[INFO]
[INFO] ------------< hbcha0916.tistory.com:nifi-hbcha-processors >-------------
[INFO] Building nifi-hbcha-processors 1.0-SNAPSHOT                        [2/3]
[INFO]   from nifi-hbcha-processors\pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- clean:3.2.0:clean (default-clean) @ nifi-hbcha-processors ---
[INFO] Deleting C:\Users\user\my-test-processor\nifi-hbcha-processors\target
[INFO]
[INFO] --- enforcer:3.2.1:enforce (enforce-maven-version) @ nifi-hbcha-processors ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.BanDuplicatePomDependencyVersions passed
[INFO] Rule 1: org.apache.maven.enforcer.rules.RequireSameVersions passed
[INFO] Rule 3: org.apache.maven.enforcer.rules.dependency.RequireReleaseDeps passed
[INFO] Rule 4: org.apache.maven.enforcer.rules.dependency.BannedDependencies passed
[INFO]
[INFO] --- enforcer:3.2.1:enforce (enforce-java-version) @ nifi-hbcha-processors ---
[INFO]
[INFO] --- buildnumber:3.0.0:create (default) @ nifi-hbcha-processors ---
[INFO]
[INFO] --- remote-resources:1.7.0:process (process-resource-bundles) @ nifi-hbcha-processors ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- resources:3.3.0:resources (default-resources) @ nifi-hbcha-processors ---
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.10.1:compile (default-compile) @ nifi-hbcha-processors ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to C:\Users\user\my-test-processor\nifi-hbcha-processors\target\classes
[INFO]
[INFO] --- resources:3.3.0:testResources (default-testResources) @ nifi-hbcha-processors ---
[INFO] skip non existing resourceDirectory C:\Users\user\my-test-processor\nifi-hbcha-processors\src\test\resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.10.1:testCompile (default-testCompile) @ nifi-hbcha-processors ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to C:\Users\user\my-test-processor\nifi-hbcha-processors\target\test-classes
[INFO]
[INFO] --- compiler:3.10.1:testCompile (groovy-tests) @ nifi-hbcha-processors ---
[INFO] Changes detected - recompiling the module!
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- surefire:3.0.0-M8:test (default-test) @ nifi-hbcha-processors ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running myprocessor.MyProcessorTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.23 s - in myprocessor.MyProcessorTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- jar:3.3.0:jar (default-jar) @ nifi-hbcha-processors ---
[INFO] Building jar: C:\Users\user\my-test-processor\nifi-hbcha-processors\target\nifi-hbcha-processors-1.0-SNAPSHOT.jar
[INFO]
[INFO] --- site:3.12.1:attach-descriptor (attach-descriptor) @ nifi-hbcha-processors ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- install:3.1.0:install (default-install) @ nifi-hbcha-processors ---
[INFO] Installing C:\Users\user\my-test-processor\nifi-hbcha-processors\pom.xml to C:\Users\user\.m2\repository\hbcha0916\tistory\com\nifi-hbcha-processors\1.0-SNAPSHOT\nifi-hbcha-processors-1.0-SNAPSHOT.pom
[INFO] Installing C:\Users\user\my-test-processor\nifi-hbcha-processors\target\nifi-hbcha-processors-1.0-SNAPSHOT.jar to C:\Users\user\.m2\repository\hbcha0916\tistory\com\nifi-hbcha-processors\1.0-SNAPSHOT\nifi-hbcha-processors-1.0-SNAPSHOT.jar
[INFO]
[INFO] ----------------< hbcha0916.tistory.com:nifi-hbcha-nar >----------------
[INFO] Building nifi-hbcha-nar 1.0-SNAPSHOT                               [3/3]
[INFO]   from nifi-hbcha-nar\pom.xml
[INFO] --------------------------------[ nar ]---------------------------------
[INFO]
[INFO] --- clean:3.2.0:clean (default-clean) @ nifi-hbcha-nar ---
[INFO] Deleting C:\Users\user\my-test-processor\nifi-hbcha-nar\target
[INFO]
[INFO] --- enforcer:3.2.1:enforce (enforce-maven-version) @ nifi-hbcha-nar ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.BanDuplicatePomDependencyVersions passed
[INFO] Rule 1: org.apache.maven.enforcer.rules.RequireSameVersions passed
[INFO] Rule 4: org.apache.maven.enforcer.rules.dependency.BannedDependencies passed
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for my-test-processor 1.0-SNAPSHOT:
[INFO]
[INFO] my-test-processor .................................. SUCCESS [  1.536 s]
[INFO] nifi-hbcha-processors .............................. SUCCESS [  3.954 s]
[INFO] nifi-hbcha-nar ..................................... FAILURE [  0.042 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  6.784 s
[INFO] Finished at: 2023-05-31T14:20:29+09:00
[INFO] ------------------------------------------------------------------------
[WARNING]
[WARNING] Plugin validation issues were detected in 7 plugin(s)
[WARNING]
[WARNING]  * org.apache.maven.plugins:maven-site-plugin:3.12.1
[WARNING]  * org.codehaus.mojo:buildnumber-maven-plugin:3.0.0
[WARNING]  * org.apache.maven.plugins:maven-compiler-plugin:3.10.1
[WARNING]  * org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M8
[WARNING]  * org.apache.maven.plugins:maven-remote-resources-plugin:1.7.0
[WARNING]  * org.apache.nifi:nifi-nar-maven-plugin:1.4.0
[WARNING]  * org.apache.maven.plugins:maven-resources-plugin:3.3.0
[WARNING]
[WARNING] For more or less details, use 'maven.plugin.validation' property with one of the values (case insensitive): [BRIEF, DEFAULT, VERBOSE]
[WARNING]
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.2.1:enforce (enforce-maven-version) on project nifi-hbcha-nar:
[ERROR] Rule 3: org.apache.maven.enforcer.rules.dependency.RequireReleaseDeps failed with message:
[ERROR] Dependencies outside of Apache NiFi must not use SNAPSHOT versions
[ERROR] hbcha0916.tistory.com:nifi-hbcha-nar:nar:1.0-SNAPSHOT
[ERROR]    hbcha0916.tistory.com:nifi-hbcha-processors:jar:1.0-SNAPSHOT <--- is not a release dependency
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :nifi-hbcha-nar

이런.. 빌드에 실패했다..

jar:1.0-SNAPSHOT 오류 해결방법

터미널에서 프로젝트의 최상위 디렉토리 ${ArtifactID}로 이동한다.
나 같은 경우는 (C:\Users\user\my-test-processor) 디렉터리이다.

아래 명령어로 "pom.xml" 위치를 찾는다.

tree /f

#결과
폴더 PATH의 목록입니다.
볼륨 일련 번호는 36DD-7C38입니다.
C:.
│  pom.xml
│
├─nifi-hbcha-nar
│      pom.xml
│
├─nifi-hbcha-processors
│  │  pom.xml
│  │
│  ├─src
│  │  ├─main
│  │  │  ├─java
│  │  │  │  └─myprocessor
│  │  │  │          MyProcessor.java
│  │  │  │
│  │  │  └─resources
│  │  │      └─META-INF
│  │  │          └─services
│  │  │                  org.apache.nifi.processor.Processor
│  │  │
│  │  └─test
│  │      └─java
│  │          └─myprocessor
│  │                  MyProcessorTest.java
│  │
│  └─target
│      │  .plxarc
│      │  nifi-hbcha-processors-1.0-SNAPSHOT.jar
│      │
│      ├─classes
│      │  ├─META-INF
│      │  │  │  DEPENDENCIES
│      │  │  │  LICENSE
│      │  │  │  NOTICE
│      │  │  │
│      │  │  └─services
│      │  │          org.apache.nifi.processor.Processor
│      │  │
│      │  └─myprocessor
│      │          MyProcessor.class
│      │
│      ├─generated-sources
│      │  └─annotations
│      ├─generated-test-sources
│      │  └─test-annotations
│      ├─maven-archiver
│      │      pom.properties
│      │
│      ├─maven-shared-archive-resources
│      │  └─META-INF
│      │          DEPENDENCIES
│      │          LICENSE
│      │          NOTICE
│      │
│      ├─maven-status
│      │  └─maven-compiler-plugin
│      │      ├─compile
│      │      │  └─default-compile
│      │      │          createdFiles.lst
│      │      │          inputFiles.lst
│      │      │
│      │      └─testCompile
│      │          ├─default-testCompile
│      │          │      createdFiles.lst
│      │          │      inputFiles.lst
│      │          │
│      │          └─groovy-tests
│      │                  createdFiles.lst
│      │                  inputFiles.lst
│      │
│      ├─surefire-reports
│      │      myprocessor.MyProcessorTest.txt
│      │      TEST-myprocessor.MyProcessorTest.xml
│      │
│      └─test-classes
│          ├─META-INF
│          │      DEPENDENCIES
│          │      LICENSE
│          │      NOTICE
│          │
│          └─myprocessor
│                  MyProcessorTest.class
│
└─target
    │  .plxarc
    │
    └─maven-shared-archive-resources
        └─META-INF
                DEPENDENCIES
                LICENSE
                NOTICE

총 3개의 pom을 찾았다.

모든 3개의 pom.xml 에서 `<version>` 태그에서 "X.X-SNAPSHOT"에서 "-SNAPSHOT"을 지워준다.
(나 같은 경우에는 "1.0-SNAPSHOT"로 나와있고 이것을 "1.0"으로만 바꿔줬다.)

다 바꿔주고 다시 빌드해 보자.

프로젝트 빌드 - 성공

위 "프로젝트 빌드" 부분을 따라한 다음 성공한 부분이다.

PS C:\Users\user\my-test-processor> mvn clean install
[INFO] Scanning for projects...
[INFO] Inspecting build with total of 3 modules...
[INFO] Installing Nexus Staging features:
[INFO]   ... total of 3 executions of maven-deploy-plugin replaced with nexus-staging-maven-plugin
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] my-test-processor                                                  [pom]
[INFO] nifi-hbcha-processors                                              [jar]
[INFO] nifi-hbcha-nar                                                     [nar]
[INFO]
[INFO] --------------< hbcha0916.tistory.com:my-test-processor >---------------
[INFO] Building my-test-processor 1.0                                     [1/3]
[INFO]   from pom.xml
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- clean:3.2.0:clean (default-clean) @ my-test-processor ---
[INFO] Deleting C:\Users\user\my-test-processor\target
[INFO]
[INFO] --- enforcer:3.2.1:enforce (enforce-maven-version) @ my-test-processor ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.BanDuplicatePomDependencyVersions passed
[INFO] Rule 1: org.apache.maven.enforcer.rules.RequireSameVersions passed
[INFO] Rule 2: org.apache.maven.enforcer.rules.version.RequireMavenVersion passed
[INFO] Rule 3: org.apache.maven.enforcer.rules.dependency.RequireReleaseDeps passed
[INFO] Rule 4: org.apache.maven.enforcer.rules.dependency.BannedDependencies passed
[INFO]
[INFO] --- enforcer:3.2.1:enforce (enforce-java-version) @ my-test-processor ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.version.RequireJavaVersion passed
[INFO]
[INFO] --- buildnumber:3.0.0:create (default) @ my-test-processor ---
[INFO]
[INFO] --- remote-resources:1.7.0:process (process-resource-bundles) @ my-test-processor ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- compiler:3.10.1:testCompile (groovy-tests) @ my-test-processor ---
[INFO] No sources to compile
[INFO]
[INFO] --- site:3.12.1:attach-descriptor (attach-descriptor) @ my-test-processor ---
[INFO] No site descriptor found: nothing to attach.
[INFO]
[INFO] --- install:3.1.0:install (default-install) @ my-test-processor ---
[INFO] Installing C:\Users\user\my-test-processor\pom.xml to C:\Users\user\.m2\repository\hbcha0916\tistory\com\my-test-processor\1.0\my-test-processor-1.0.pom
[INFO]
[INFO] ------------< hbcha0916.tistory.com:nifi-hbcha-processors >-------------
[INFO] Building nifi-hbcha-processors 1.0                                 [2/3]
[INFO]   from nifi-hbcha-processors\pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- clean:3.2.0:clean (default-clean) @ nifi-hbcha-processors ---
[INFO] Deleting C:\Users\user\my-test-processor\nifi-hbcha-processors\target
[INFO]
[INFO] --- enforcer:3.2.1:enforce (enforce-maven-version) @ nifi-hbcha-processors ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.BanDuplicatePomDependencyVersions passed
[INFO] Rule 1: org.apache.maven.enforcer.rules.RequireSameVersions passed
[INFO] Rule 3: org.apache.maven.enforcer.rules.dependency.RequireReleaseDeps passed
[INFO] Rule 4: org.apache.maven.enforcer.rules.dependency.BannedDependencies passed
[INFO]
[INFO] --- enforcer:3.2.1:enforce (enforce-java-version) @ nifi-hbcha-processors ---
[INFO]
[INFO] --- buildnumber:3.0.0:create (default) @ nifi-hbcha-processors ---
[INFO]
[INFO] --- remote-resources:1.7.0:process (process-resource-bundles) @ nifi-hbcha-processors ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- resources:3.3.0:resources (default-resources) @ nifi-hbcha-processors ---
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.10.1:compile (default-compile) @ nifi-hbcha-processors ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to C:\Users\user\my-test-processor\nifi-hbcha-processors\target\classes
[INFO]
[INFO] --- resources:3.3.0:testResources (default-testResources) @ nifi-hbcha-processors ---
[INFO] skip non existing resourceDirectory C:\Users\user\my-test-processor\nifi-hbcha-processors\src\test\resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.10.1:testCompile (default-testCompile) @ nifi-hbcha-processors ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to C:\Users\user\my-test-processor\nifi-hbcha-processors\target\test-classes
[INFO]
[INFO] --- compiler:3.10.1:testCompile (groovy-tests) @ nifi-hbcha-processors ---
[INFO] Changes detected - recompiling the module!
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- surefire:3.0.0-M8:test (default-test) @ nifi-hbcha-processors ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running myprocessor.MyProcessorTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.247 s - in myprocessor.MyProcessorTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- jar:3.3.0:jar (default-jar) @ nifi-hbcha-processors ---
[INFO] Building jar: C:\Users\user\my-test-processor\nifi-hbcha-processors\target\nifi-hbcha-processors-1.0.jar
[INFO]
[INFO] --- site:3.12.1:attach-descriptor (attach-descriptor) @ nifi-hbcha-processors ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- install:3.1.0:install (default-install) @ nifi-hbcha-processors ---
[INFO] Installing C:\Users\user\my-test-processor\nifi-hbcha-processors\pom.xml to C:\Users\user\.m2\repository\hbcha0916\tistory\com\nifi-hbcha-processors\1.0\nifi-hbcha-processors-1.0.pom
[INFO] Installing C:\Users\user\my-test-processor\nifi-hbcha-processors\target\nifi-hbcha-processors-1.0.jar to C:\Users\user\.m2\repository\hbcha0916\tistory\com\nifi-hbcha-processors\1.0\nifi-hbcha-processors-1.0.jar
[INFO]
[INFO] ----------------< hbcha0916.tistory.com:nifi-hbcha-nar >----------------
[INFO] Building nifi-hbcha-nar 1.0                                        [3/3]
[INFO]   from nifi-hbcha-nar\pom.xml
[INFO] --------------------------------[ nar ]---------------------------------
[INFO]
[INFO] --- clean:3.2.0:clean (default-clean) @ nifi-hbcha-nar ---
[INFO]
[INFO] --- enforcer:3.2.1:enforce (enforce-maven-version) @ nifi-hbcha-nar ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.BanDuplicatePomDependencyVersions passed
[INFO] Rule 1: org.apache.maven.enforcer.rules.RequireSameVersions passed
[INFO] Rule 3: org.apache.maven.enforcer.rules.dependency.RequireReleaseDeps passed
[INFO] Rule 4: org.apache.maven.enforcer.rules.dependency.BannedDependencies passed
[INFO]
[INFO] --- enforcer:3.2.1:enforce (enforce-java-version) @ nifi-hbcha-nar ---
[INFO]
[INFO] --- buildnumber:3.0.0:create (default) @ nifi-hbcha-nar ---
[INFO]
[INFO] --- remote-resources:1.7.0:process (process-resource-bundles) @ nifi-hbcha-nar ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- resources:3.3.0:resources (default-resources) @ nifi-hbcha-nar ---
[INFO] skip non existing resourceDirectory C:\Users\user\my-test-processor\nifi-hbcha-nar\src\main\resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.10.1:compile (default-compile) @ nifi-hbcha-nar ---
[INFO] No sources to compile
[INFO]
[INFO] --- resources:3.3.0:testResources (default-testResources) @ nifi-hbcha-nar ---
[INFO] skip non existing resourceDirectory C:\Users\user\my-test-processor\nifi-hbcha-nar\src\test\resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.10.1:testCompile (default-testCompile) @ nifi-hbcha-nar ---
[INFO] No sources to compile
[INFO]
[INFO] --- compiler:3.10.1:testCompile (groovy-tests) @ nifi-hbcha-nar ---
[INFO] No sources to compile
[INFO]
[INFO] --- surefire:3.0.0-M8:test (default-test) @ nifi-hbcha-nar ---
[INFO]
[INFO] --- nifi-nar:1.4.0:nar (default-nar) @ nifi-hbcha-nar ---
[INFO] Copying nifi-hbcha-processors-1.0.jar to C:\Users\user\my-test-processor\nifi-hbcha-nar\target\classes\META-INF\bundled-dependencies\nifi-hbcha-processors-1.0.jar
[INFO] Copying jakarta.activation-1.2.2.jar to C:\Users\user\my-test-processor\nifi-hbcha-nar\target\classes\META-INF\bundled-dependencies\jakarta.activation-1.2.2.jar
[INFO] Copying jaxb-runtime-2.3.5.jar to C:\Users\user\my-test-processor\nifi-hbcha-nar\target\classes\META-INF\bundled-dependencies\jaxb-runtime-2.3.5.jar
[INFO] Copying nifi-utils-1.20.0.jar to C:\Users\user\my-test-processor\nifi-hbcha-nar\target\classes\META-INF\bundled-dependencies\nifi-utils-1.20.0.jar
[INFO] Copying jakarta.xml.bind-api-2.3.3.jar to C:\Users\user\my-test-processor\nifi-hbcha-nar\target\classes\META-INF\bundled-dependencies\jakarta.xml.bind-api-2.3.3.jar
[INFO] Copying txw2-2.3.5.jar to C:\Users\user\my-test-processor\nifi-hbcha-nar\target\classes\META-INF\bundled-dependencies\txw2-2.3.5.jar
[INFO] Copying istack-commons-runtime-3.0.12.jar to C:\Users\user\my-test-processor\nifi-hbcha-nar\target\classes\META-INF\bundled-dependencies\istack-commons-runtime-3.0.12.jar
[INFO] Copying jakarta.activation-api-1.2.2.jar to C:\Users\user\my-test-processor\nifi-hbcha-nar\target\classes\META-INF\bundled-dependencies\jakarta.activation-api-1.2.2.jar
[INFO] Generating documentation for NiFi extensions in the NAR...
[INFO] Found a dependency on version 1.20.0 of NiFi API
[INFO] Building jar: C:\Users\user\my-test-processor\nifi-hbcha-nar\target\nifi-hbcha-nar-1.0.nar
[INFO]
[INFO] --- site:3.12.1:attach-descriptor (attach-descriptor) @ nifi-hbcha-nar ---
[INFO] Skipping because packaging 'nar' is not pom.
[INFO]
[INFO] --- install:3.1.0:install (default-install) @ nifi-hbcha-nar ---
[INFO] Installing C:\Users\user\my-test-processor\nifi-hbcha-nar\pom.xml to C:\Users\user\.m2\repository\hbcha0916\tistory\com\nifi-hbcha-nar\1.0\nifi-hbcha-nar-1.0.pom
[INFO] Installing C:\Users\user\my-test-processor\nifi-hbcha-nar\target\nifi-hbcha-nar-1.0.nar to C:\Users\user\.m2\repository\hbcha0916\tistory\com\nifi-hbcha-nar\1.0\nifi-hbcha-nar-1.0.nar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for my-test-processor 1.0:
[INFO]
[INFO] my-test-processor .................................. SUCCESS [  1.497 s]
[INFO] nifi-hbcha-processors .............................. SUCCESS [  3.759 s]
[INFO] nifi-hbcha-nar ..................................... SUCCESS [  0.729 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  7.275 s
[INFO] Finished at: 2023-05-31T14:32:42+09:00
[INFO] ------------------------------------------------------------------------
[WARNING]
[WARNING] Plugin validation issues were detected in 7 plugin(s)
[WARNING]
[WARNING]  * org.apache.maven.plugins:maven-site-plugin:3.12.1
[WARNING]  * org.codehaus.mojo:buildnumber-maven-plugin:3.0.0
[WARNING]  * org.apache.maven.plugins:maven-compiler-plugin:3.10.1
[WARNING]  * org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M8
[WARNING]  * org.apache.maven.plugins:maven-remote-resources-plugin:1.7.0
[WARNING]  * org.apache.nifi:nifi-nar-maven-plugin:1.4.0
[WARNING]  * org.apache.maven.plugins:maven-resources-plugin:3.3.0
[WARNING]
[WARNING] For more or less details, use 'maven.plugin.validation' property with one of the values (case insensitive): [BRIEF, DEFAULT, VERBOSE]
[WARNING]

우리가 봐야 할 것은 [INFO] BUILD SUCCESS!!

적용하기

빌드에 성공했으면 ${ArtifactID} > nifi - ${ArtifactBaseName} - nar > target 디렉터리가 생겼다.

(위 예론 "C:\Users\user\my-test-processor\nifi-hbcha-nar\target" 디렉터리이다.)

해당 디렉토리에 "nifi - ${ArtifactBaseName} - nar . nar" 파일이 생성된 것을 알 수 있다.

해당 .nar 파일을 $NIFI_HOME/lib 디렉토리에 복사 후 NiFi 재부팅한다.

성공하면 아래와 같은 프로세서가 생긴 것을 확인할 수 있다.

물론 동작하진 않는다.(properties와relationship, 그리고 FlowFile처리 로직을 작성하지 않았기 때문이다.)

항상 만들기 전에 내가 만들려는 프로세서가 있나 자세히 검색해 보고 만들자

저작자표시 변경금지 (새창열림)

'NiFi' 카테고리의 다른 글

NiFi TailFile (1)	2024.01.17
[Python - Selenium]NiFi ExecuteStreamCommand로 웹크롤링 하기 (0)	2023.06.24
Remote Process Group 사용법 (0)	2023.05.24
NiFi parameter contexts 추출 및 불러오는 방법(백업/복원 방법) (0)	2023.05.23
NiFi TimeZone변경 (0)	2023.05.23

내 블로그 - 관리자 홈 전환	`Q` `Q`
새 글 쓰기	`W` `W`

글 수정 (권한 있는 경우)	`E` `E`
댓글 영역으로 이동	`C` `C`

이 페이지의 URL 복사	`S` `S`
맨 위로 이동	`T` `T`
티스토리 홈 이동	`H` `H`
단축키 안내	`Shift` + `/` `⇧` + `/`

NiFi Custom Processor 만들기

Custom Processor 을 만드는 이유

준비

프로젝트 생성

[샘플] 프로젝트 제작

이클립스 혹은 VScode 등 IDE에서 해당 프로젝트를 열기

프로젝트 빌드

jar:1.0-SNAPSHOT 오류 해결방법

프로젝트 빌드 - 성공

적용하기

'NiFi' 카테고리의 다른 글

Custom Processor 을 만드는 이유

준비

프로젝트 생성

[샘플] 프로젝트 제작

이클립스 혹은 VScode 등 IDE에서 해당 프로젝트를 열기

프로젝트 빌드

jar:1.0-SNAPSHOT 오류 해결방법

프로젝트 빌드 - 성공

적용하기

'NiFi' 카테고리의 다른 글

티스토리툴바

단축키

내 블로그

블로그 게시글

모든 영역